Random Word Recognition Chart Helps Scotoma Assessment in Low Vision

ABSTRACT

Purpose

To evaluate the use of SKread, a vision test based on random word sequences that prevents the prediction of upcoming words by linguistic criteria and is simple to score in a clinical setting.

Methods

SKread combines the standardized format of the MNread test with sequences of random words and letters like the Pepper Visual Skills for Reading test. A total of 231 subjects (aged 16 to 97 years) participated. We report data from 136 eyes of subjects with a maculopathy and 65 with normal or near-normal vision. Test reliability was investigated on an additional 30 eye-healthy subjects. We tested visual acuity and reading performance for continuous text and random words monocularly. Reading speed and all errors made are reported.

Results

Reading speed was always higher for continuous text than for random word sequences, even in normally sighted subjects for whom the median reading times per paragraph were 2.4 s (MNread) vs. 6.8 s (SKread). In patients with maculopathies, the medians were 4.2 s vs. 12.25 s. These differences were statistically significant. Number and type of errors made depended only negligibly on age and visual acuity. Patients with a dense scotoma right of fixation made more “right errors” by missing letters at the end of words, whereas those with a scotoma left of fixation made more “left errors” by missing letters at the beginning of words. The SKread test showed good test-retest repeatability.

Conclusions

The unpredictability of random word and letter sequences renders reading performance highly dependent on eyesight and less dependent on reading skill and educational level. Recurrent right or left errors can indicate the presence and location of a scotoma without expensive equipment. This knowledge can be used to teach patients about how the scotoma can interfere with their vision.

A limiting factor in reading with low vision can be visual acuity that is routinely tested by the MNread test.1 Loss of the central field and interference by a central scotoma can further impair word recognition and localization of the following line. The goal of this work was to evaluate the use of the SKread test in exposing vision problems that may impair reading. The test requires recognition of printed words but does not allow prediction of the next word or letter. The concept is similar to the existing Pepper test2 but in a format designed for straightforward analysis in the clinic, as exemplified by MNread. The resulting SKread test minimizes the influence of linguistics, reading experience, and education level on test outcome.

Background

Printed words as test material have been common in vision care for at least 150 years because of the importance of reading in cultural and educational contexts.3–6 Two aspects of vision are most frequently tested6:

1. reading acuity and critical print size and

2. reading performance, that is, speed and accuracy

The former stresses the point that reading depends on physical dimensions, for example, letter size and contrast.7 In countries with a predominantly English-speaking population, the standard tools are the MNread test charts.1

The latter aspect stresses the fact that reading also depends on cognitive, educational, and linguistic components.2,8 The two most important elements of the linguistics of reading are syntax and semantics.9 Both can transform a random string of words into continuous text, which increases reading speed and fluency by making contextual and lexical inference possible. Thus, they allow associating a probability with each word that determines the likelihood that a certain word may follow.10–13 The ability to use this prediction determines reading speed and is the basis of the difference between fluent and nonfluent reading.

Theoretical Considerations

If a sequence of words loses the syntactic and semantic coherence, reading becomes more difficult and slows down.14,15 In the simplest case, reading can be performed letter-by-letter, which is very slow because only one letter is recognized at a time. Examples are children who just start to learn how to read or patients with a ring scotoma and a small central seeing island.15,16

Recognizing a group of letters as a word presents an advantage. Consequently, reading word-by-word is faster because it allows a limited degree of prediction of upcoming words. For instance, the ascenders or descenders of letters in an upcoming word can function as guides for rapid recognition of a word shape, as in “reap” vs. “read.” There is evidence that word recognition involves visual cortex that is also sensitive to other shapes as well as colors and faces.17

A further mechanism can contribute to predictions of longer words where recognizing a few letters in the beginning may reduce the range of possibilities to just a few words by lexical inference.10,13,18 The fastest reading speed can be achieved with the highest degree of prediction by using the relationships between words (linguistic inference). This does not even require precise recognition of an upcoming word. Consider the common phrase “for all intents and purposes,” where “purposes” is very predictable once the first three words are recognized. In contrast, the phrase “for all intents and programs” is highly unlikely. Interactions of various linguistic elements are so complex that reading has been used to estimate levels of intelligence.19

There is evidence that prediction of upcoming words during reading is used to determine the landing positions of reading eye movements.20 In real-life horizontal reading, upcoming words are visible as long as a horizontal “perceptual span” of 15 to 20 letters is not obstructed.21–26 The functional importance of this span becomes evident by the poor reading performance of patients with restricted visual fields 27,28 and with central scotomata.29,30

Two established vision tests in the English language focus on the difference between reading with or without linguistic inference. The MNread test uses simple sentences with intact grammar and meaning. Each is composed of 60 letters, including spaces, that are printed on three lines.1 The standardized format allows quick comparisons between reading performance of paragraphs printed in different sizes. It is the preferred instrument to determine reading speed and critical print size and is a well-used clinical tool to evaluate reading performance in low vision.31 Note that the original publication also explored using sequences of unrelated words.1

On the other hand, the Pepper Visual Skills for Reading (VSR) test uses random sequences of words and single letters in a paragraph of 13 lines, which prevents prediction of the next word based on the occurrence of the previous one.2,32 The lines are of different lengths (29 to 45 letters, including spaces). This test allows differentiating nine error types so that accurate scoring requires off-line data analysis from an audio recording. The special feature of this test is that it does not allow prediction based on linguistic inference. Its disadvantage is that the results need to be analyzed and quantified off-line, which is too time-consuming in a clinical setting.

To overcome this disadvantage, we applied the proven standardized format of the MNread test to groups of unrelated words and letters, similar to the content of the VSR test to create the SKread test. The gained advantage is that, during the test, the examiner can annotate a score sheet showing the same print material as the test charts without the need for off-line data analysis.

For this study, we related data from the de facto standard MNread test to those from the SKread test to investigate the following questions:

1. In what way do the differences in test material influence reading speed and error rate?

2. Does the presence and location of a central scotoma in patients with a maculopathy influence the characteristics of reading errors?

A subset of data presented here has been previously published in conference proceedings.33–36

METHODS

Subjects

A total of 231 subjects were recruited for these experiments. We monocularly tested 201 eyes of subjects with either a maculopathy (N = 136) or normal or near-normal vision (N = 65), one eye each. The remaining 30 subjects were eye healthy and were tested binocularly only to investigate SKread test-retest reliability.

Eyes that were not able to read the largest text in MNread (8M) were excluded. All subjects spoke English, although English was the first language in only 84.5% of them. Subjects who showed conspicuous deficits regarding their ability to read and speak English were not included in this study.

The ages of subjects with the 136 eyes with a maculopathy ranged from 19 to 97 years (median, 81.0 years; interquartile range [IQR], 12.0). The visual acuity (VA) range was 20/13 to 20/1000 (median, 20/100; IQR, 20/140.5).

The ages of the 65 subjects whose healthy eye we tested ranged from 28 to 85 years (median, 56.0 years; IQR, 20.25), and visual acuities of the tested eyes varied between 20/13 and 20/25 (median, 20/20; IQR, 20/5.0).

Test-retest reliability of the SKread was tested in an additional group of 30 eye-healthy subjects who were all native English speakers. None of the authors were included in this group. Ages ranged from 22 to 76 years (median, 57.5 years; IQR, 21.0), and visual acuity was 20/30 or better.

All subjects used their habitual correction for the viewing distance of 40 cm if necessary.

Stimuli

We tested 1. visual acuity (ETDRS chart)37; 2. continuous text reading (MNread chart)1; and 3. random word reading38 like in the Pepper test2,32 (SKread chart),33–36 always in the same order. The SKread chart uses the same format as the MNread chart (60 characters per three-line paragraph of print for each font size). However, instead of sentences, it uses single letters and random common words of two to six letters length (mean, 2.95 letters) to prevent contextual, grammatical inference, or correction of errors like the Pepper VRS test.2 The words were selected from the word frequency list of the 2000 most common words in American English (http://en.wiktionary.org) based on a corpus of more than nine million words. Here are two examples of paragraphs of SKread material:

Reading times were measured by a stopwatch in seconds per paragraph. We preferred measuring the reading time rather than reading speed in words per minute because of better comparability of the results. We considered it inappropriate to treat the single letters in the SKread as “words.” All tests were performed at a viewing distance of 40 cm.

We also performed microperimetry by scanning laser ophthalmoscope ([SLO] model 101; Rodenstock, Munich, Germany) in 86 patients with a maculopathy to explore the functional status of the retina and found that 72 had a dense central scotoma.

The criterion for the definition of “dense” was that a small flashing light dot of 200 ms duration at maximum brightness (HeNe laser power level 2 = 0.9 μWatt, equivalent to 180 cd/m²) could not be detected. Topographic mistakes caused by involuntary eye movements and fixational imprecision39 were prevented by the use of a special software (“Smart Micro-Perimetry,” MMTest, San Francisco, CA) that scanned the stimuli onto the retina in a gaze-contingent manner.40 Thus, the retinal position to be stimulated wandered with the eye if unintended eye movements occurred. The fixation target was a cross of variable size and stroke width. Its size could be adjusted by the microperimetry software, so that each patient had to confirm that he or she actually saw the fixation mark. When he or she did, he or she was asked to look at it as steadily as possible.

The experiments were in compliance with the tenets of the Declaration of Helsinki.

Data Analysis

For concurrent rating of test performance, the examiner used a score sheet showing the same print material as on the test chart, but all paragraphs were printed at the same size. The overall number of mistakes was noted for all paragraphs in either test. We also differentiated them into “right mistakes” (R-MIS, e.g., “drab” instead of “drag”) and “left mistakes” (L-MIS, e.g., “drain” instead of “train”).

We used StatView (Abacus Software, Berkeley, CA) and Excel (Microsoft, Redmond, WA) for statistical analysis. Nonparametric methods were used to prevent having to assume that data were normally distributed. Group differences were assessed by the Mann-Whitney U (MWU) test. Associations and correlations were expressed as the coefficient of determination r², which shows to what degree the variability in a data set can be accounted for by the model used.

RESULTS

Test-Retest Reliability

Each of the 30 control-only subjects read five paragraphs of the SKread charts 1 and 2 within a few minutes. We used the mean reading time for each set of five paragraphs to characterize their performance. For each subject, the print sizes of the paragraphs read were the same in both readings.

Test reliability was first expressed as the correlation coefficient between the two measurements, which was r = 0.925, and thus the coefficient of determination r² was 0.855. We also calculated the coefficient of repeatability CR41–43 as 1.96 × SD of the mean differences between the first and second reading, which was 0.567 (±0.434 SD). Hence, CR = 1.96 × 0.434 = 0.851, which is 9% of the mean reading time 9.458 s.

Reading Speed

There is a fundamental difference in the degree of difficulty between reading continuous text and random words and letters. Our findings show that reading speed was always higher for continuous text, even for normally sighted individuals.

The median time it took for 65 normally sighted eyes to read the largest size (8M) of MNread was 2.4 s (range, 1.5 to 6.8 s; IQR, 0.70) but, for SKread, it took 6.8 s (range, 4.0 to 15.8 s; IQR, 2.7). Accordingly, median reading speed in characters per second (CPS) were 25.0 CPS (IQR, 7.8; range, 8.8 to 40.0 CPS) for MNread and 8.8 CPS (IQR, 3.7; range, 3.8 to 15.0 CPS) for SKread. Both speed differences between the tests were statistically highly significant (MWU test, U[s] = 109 and U[CPS] = 33; p < 0.0001). We use CPS as a measure of reading speed to allow a fair comparison because both tests share one character as the smallest unit, whereas a single character in SKread should not be treated as “words.”

Note that MNread and SKread showed only a minimal dependence on subject age or on visual acuity, as the coefficients of determination for these relationships were small (r² < 0.15).

In the 136 eyes with a maculopathy, reading times for 8M print ranged from 1.70 to 67.3 s (MNread) and from 5.0 to 204.0 s (SKread). The median reading times were 4.2 s (IQR, 5.35) and 12.25 s (IQR, 15.20), respectively, that is, a factor of about 3 longer in the SKread.

The results for maximum reading speed were similar: medians for MNread were 14.3 CPS (IQR, 11.2; range, 0.9 to 35.3 CPS) and for SKread 4.9 CPS (IQR, 4.0; range, 0.3 to 12.0 CPS), again a factor of 3. Considering only performance at a print size of 8M, the differences between the two tests in reading time as well as maximum reading speed were statistically highly significant (MWU test, U[s] = 1457.5 and U[CPS] = 1357; p < 0.0001). A summary of these results is shown in Table 1.

TABLE 1.

Differences between performance levels in the MNread and SKread tests for subjects with healthy eyes and with a maculopathy

Mistakes Made

The difference between the degree of predictability of continuous text vs. random words was already evident in the performance of normally sighted subjects. As expected, they made few mistakes, but the difference between the two test materials was conspicuous: The 65 normally sighted eyes (subjects) read altogether 718 paragraphs of MNread material and made 21 mistakes, which is 0.029 mistakes per paragraph. Note that these 21 mistakes were made by only 11 of 65 subjects.

In contrast, when the same group read 723 paragraphs of SKread material, they made 317 mistakes (0.438 per paragraph), which is about 15 times as many as for MNread. In this case, the 317 mistakes were made by 53 of 65 subjects (see Table 2).

TABLE 2.

Number of mistakes made per paragraph during reading the MNread and SKread charts (#MNread and #SKread)

The influence of the test material was more pronounced in the 136 eyes of patients with maculopathies and potentially a dense central scotoma. Because each patient could read only a limited number of paragraphs, we standardized performance by dividing the total number of mistakes by the number of paragraphs read, which gave us mistakes per paragraph. The 136 eyes with maculopathies made 0.23 mistakes per paragraph of MNread, but 1.38 mistakes per paragraph in SKread, a factor of about 6.

The number of mistakes made per paragraph again showed only a negligible dependence on subject age and visual acuity (r² < 0.1).

Microperimetry by SLO on 86 eyes with maculopathies showed that 72 of them had a dense central scotoma. These 72 eyes together read 511 paragraphs of MNread text and made a total of 152 mistakes in the process, which is about 0.3 (0.297) mistakes per paragraph. In contrast, in reading 489 paragraphs of SKread material, the same 72 eyes made 1068 mistakes, which is about 2.18 mistakes per paragraph. Thus, the eyes with a dense scotoma made 7.3 times more mistakes when reading SKread material than when reading continuous text (see Table 2).

The MWU test showed that the difference in mistakes per paragraph between MNread and SKread was statistically highly significant (U = 1917, p < 0.0001). Comparing the median values of mistakes per paragraph between SKread and MNread even yielded a factor of approximately 7.2 (1.79 vs. 0.25, respectively).

We also collected data about the spatial relations between dense scotomas and the directional characteristics of the reading mistakes. The results showed a conspicuous correlation between the position of the scotoma in the visual field and the probability of misinterpreting letters at the beginning and end of words (see Figs. 1 and 2). We disregarded scotomas lying above or below fixation because English is read horizontally, so that a scotoma above or below fixation is not likely to interfere with reading. Of the 72 eyes with a dense scotoma, 52 had a scotoma to the right of fixation (R-scotoma), that is, in the reading direction, and 36 had a scotoma to the left of fixation (L-scotoma). Note that these two numbers do not add up to the 72 eyes examined because several patients had a scotoma on both sides of fixation. Subjects with an R-scotoma made 220 R-mistakes and only eight L-mistakes, a ratio of 27.5. Conversely, subjects with an L-scotoma made 100 L-mistakes and only 16 R-mistakes, a ratio of 6.25.

FIGURE 1.

Schematic drawing of the patient’s visual field with a scotoma to the right of the PRL (shown by a circle). The scotoma covers the last letter, so that the word “theme” is read as “them.”

FIGURE 2.

Schematic drawing of the patient’s visual field with a scotoma to the left of the PRL (shown by a circle). The scotoma covers the first letter, so that the word “stone” is read as “stone.”

The probability to make a right/left mistake can be calculated as the frequency, with which the right/left mistakes occur divided by the total number of mistakes.

The probabilities to make left and right mistakes with a left and right scotoma were compared using the MWU test. In eyes with an L-scotoma, the two values differed by 0.072, but this difference was statistically not significant. On the other hand, for eyes with an R-scotoma, these probabilities differed by 0.318, and this difference was statistically highly significant (U = 268; p < 0.0001).

The condition of these patients and the differentiation into R-mistakes vs. L-mistakes also allowed us to compare the mistakes made in the MNread as well as the SKread test. The 86 patients read 664 paragraphs of MNread and made 158 mistakes doing it (i.e., 0.238 per paragraph). As expected, many of the patients made no mistake at all while reading MNread, that is, 25 of 86, which are about one-third (29%). Thus, the 158 mistakes were made by the rest of the 61 patients. Only about one-half of the overall number of mistakes could be identified as R-mistakes (56) or L-mistakes (30). The ratio of R-/L-mistakes was 1.87.

On the other hand, while reading 635 paragraphs of SKread, only one was able to do it without mistake, whereas 85 of 86 patients made 1074 mistakes (i.e., 1.69 per paragraph). Of those, 433 could be identified as R-mistakes (284) or L-mistakes (149). The ratio of R-/L-mistakes was 1.91. Counting all mistakes per paragraph, these patients made 7.1 times as many mistakes using SKread than reading MNread and five times as many directional (R- and L-) mistakes.

Our list of patients also contained some with a scotoma on the right and the left of fixation, so that they had reasons to make R- as well as L-mistakes. We examined 19 such cases and found that only four of them actually showed R- and L-mistakes. On further examination, they turned out to be the ones with the largest estimated horizontal scotoma diameter (10 to 20 degrees in the SLO images), whereas all others had only diameters of 2 to 8 degrees. Further analysis indicated that each of these four patients had a ring scotoma.

To create the cleanest conditions, we repeated the same error analysis of MNread and SKread after excluding these 19 cases. The rest of the 67 patients read 507 paragraphs MNread and made 132 mistakes (0.26 per paragraph), with 19 of 67 making no mistakes at all. Forty-six were identified as R-mistakes and 20 as L-mistakes, with a ratio of R-/L-mistakes of 2.3. The same 67 patients read 485 paragraphs of SKread and made 852 mistakes (with the exception of one person; that is, 1.76 per paragraph). There were 213 R-mistakes and 109 L-mistakes. The ratio of R-/L-mistakes was 1.95. These results show that the presence of the small subgroup with a scotoma right and left of fixation did not substantially distort the findings.

Analysis of the data from normally sighted subjects showed that the question whether they grew up with English as their first language or not did not matter (0.426 vs. 0.44 mistakes per paragraph).

DISCUSSION

Test-Retest Reliability

Based on previous work on test-retest reliability of vision tests using continuous text and strings of random words,1,2 we had an expectation that the SKread test may also fare well in this regard.32,44 Our results confirm this expectation by a high coefficient of determination of r² = 0.855, which means that 85.5% of the data variance is accounted for by the test itself. Furthermore, the CR43 of 9% of the mean reading time is excellent and shows the high reliability of the SKread test.

Reading Speed

One of the principal findings of this research was the difference in reading speed depending on the used test material. It should be noted that Legge et al.1 found somewhat smaller differences of 15 to 30% between the MNread and random word sequences. They even found a few subjects in whom word reading was faster than sentence reading (see their fig. 3). It cannot be ruled out that this finding was influenced by the choice of low-vision patients, which might have been different in their study.

Our findings, on the other hand, resemble reading by patients with a ring scotoma, which occurs rather frequently in different types of maculopathies.45–47 The underlying mechanism has been analyzed as the relationship of the width of the visual span and reading performance.18 The differences can further be explained by the lack of prediction of the next words by removal of sentence context in the SKread test.11 This forced our subjects to decipher each piece of print in a “bottom-up” way, whether it was a word or a single letter. We conclude from this finding that anticipating the next word is an important element in enabling high reading speed and fluency for continuous text, as in the MNread test, and that both are greatly reduced if this is impossible, as in the SKread test.2,32

Furthermore, the SKread test reduces the interindividual differences between varying reading skill levels and educational status because educational status, like having command over a large vocabulary and being a well-practiced reader, does not help in recognizing unrelated words.48 Note that these conclusions are theoretically justified for subjects with normal vision as well as for patients with low vision. On a practical scale, our data did not show a sufficient degree of diversity to produce significant effects because all our subjects had between 12 and 18 years of education.

Thus, the slowing down of reading by missing context and grammar prevents prediction of upcoming words. This effect can be interpreted as an effect of missing text complexity, which has been demonstrated as a slowing factor before. 49 The underlying mechanism has also been interpreted as cognitive.50 The link to reading speed is likely made by lengthening the durations of fixations between reading saccades as a function of text complexity.18

Mistakes Made

The higher degree of challenge in the SKread test was further demonstrated by the significantly higher number of reading mistakes. Our analysis allowed the differentiation of mistakes made by eyes with a scotoma that is located either left or right of the retinal locus used for fixation, whether this was the fovea or a preferred retinal locus (PRL). The disproportionality we found between subjects with a scotoma to the right and left of fixation can tentatively be explained as induced by the directional imbalance caused by the fact that most reading saccades are going rightward. A possible conclusion from this is that, if an eye with a scotoma to the right of fixation performs a reading saccade “toward the scotoma,” the movement may not be far enough to “uncover” the entire word to be read. This conclusion relies on previous findings that have shown that the “landing places” of reading saccades are usually near the beginning of the word51 and, for short words, like the ones used in SKread, about on the third letter.52 Hence, the known proximity of a PRL to the border of a scotoma44 makes it possible that the end of even a short word might be obscured or blurred by the scotoma, so that a misinterpretation of a word can occur.

Similar reasoning can be applied to the cases of eyes with a dense scotoma left of fixation. In these cases, the saccadic landing places near the beginning of words caused the obscuring of the beginning of the word to be less likely, which is reflected by the smaller probability of left mistakes made by eyes with a dense scotoma left of fixation. This right-left asymmetry has also been found by others using the Pepper VSR test.53,54

A practical aspect of the mechanics of running the SKread test deserves to be mentioned: Keeping track of mistakes in real time while continuous text is read can be rather difficult. Because of the uniform format of the test material that we modeled after the MNread test, in combination with the considerable slowing down of reading speed, this task is actually quite feasible in the SKread test. This has an important practical consequence for the examiner because we designed the SKread protocol with the goal to make off-line analysis unnecessary. It should be mentioned, however, that a detailed investigation beyond the clinical scope would be entirely possible by recording not only the eye movements but also the voice of the reader. However, for the purposes of clinical examinations on predominantly eyes with low vision, this turned out to be unnecessary.

Because the paragraphs of both the MNread and SKread charts are printed in different letter sizes, it would be theoretically possible to assess critical print size with the SKread too. However, the process would be more time-consuming because of the slower reading speed, and the data could be distorted by the much higher number of mistakes in SKread. The results would be unrealistic because critical print size should relate directly to the ability to read continuous text in an everyday context. Hence, the MNread test is more appropriate as a standard tool for measuring critical print size. On the other hand, SKread can be more successful as a clinical tool to expose how scotoma interference to the right and left of fixation can impair reading speed and accuracy. This was shown by the comparative error analysis for both procedures. Whenever the mistakes made are of interest, the SKread test is more appropriate 1. because many subjects make no mistakes in the MNread at all, so that the sampling would be distorted, and 2. because patients with maculopathies make seven times more mistakes overall in SKread and five times more directional mistakes than in MNread.

The SKread test has other clinical applications, especially in low-vision rehabilitation. It allows testing vision in its untainted form without the interference of uncontrollable variables like current reading practice and level of education. The frequently occurring mistakes can be a first indication that a patient has a scotoma without the use of expensive equipment and time-consuming procedures like perimetry.

In addition, the frequent mistakes have a potential benefit for educating patients about how their scotoma can affect their vision. This can be necessary because many patients cannot visualize their visual field defects,55,56 and they are often surprised to see the repeated pattern of mistakes they make to one side of words. Making them aware of this connection can facilitate their rehabilitation. Furthermore, the rehabilitation process can be aided by advising therapists in what direction compensatory eye movement training may decrease reading inaccuracies.57–59

It is noteworthy that the findings of this study have recently also been tested and confirmed in other Indo-European languages including French,60 Italian,61,62 and German.63

It would be desirable to gain a better understanding of the precise interactions of scotomata on the horizontal meridian with reading saccades and random word sequences. However, because this would have far exceeded the limitations of a study in a clinical setting, we have to leave this analysis to future research.

Manfred MacKeben
The Smith-Kettlewell Eye Research Institute
2318 Fillmore Street
San Francisco, CA 94115
e-mail: mm@ski.org