Modulation of oculomotor control during reading of mirrored and inverted texts

Abstract

The interplay between cognitive and oculomotor processes during reading can be explored when the spatial layout of text deviates from the typical display. In this study, we investigate various eye-movement measures during reading of text with experimentally manipulated layout (word-wise and letter-wise mirrored-reversed text as well as inverted and scrambled text). While typical findings (e.g., longer mean fixation times, shorter mean saccades lengths) in reading manipulated texts compared to normal texts were reported in earlier work, little is known about changes of oculomotor targeting observed in within-word landing positions under the above text layouts. Here we carry out precise analyses of landing positions and find substantial changes in the so-called launch-site effect in addition to the expected overall slow-down of reading performance. Specifically, during reading of our manipulated text conditions with reversed letter order (against overall reading direction), we find a reduced launch-site effect, while in all other manipulated text conditions, we observe an increased launch-site effect. Our results clearly indicate that the oculomotor system is highly adaptive when confronted with unusual reading conditions.

Introduction

Visual acuity is greatest at the center of visual field (the fovea) and declines sharply on the periphery, which limits the information extraction process of visual input from the environment. To compensate this limitation, the eyes generate short and rapid movements, saccades, to shift the fovea to the regions of interests for high-acuity information processing¹. During reading, the eyes move forward about 6 to 7 character spaces, on average, during saccades and fixate on a word for about 200 to 250 ms to support word processing. The control of (saccadic) eye movements requires the coordination of several fundamental cognitive subsystems such as word recognition, attention², and oculomotor control³. While the cognitive system is responsible for selecting which word to be fixated next, it is the oculomotor system that is responsible for shifting the fovea to the regions of interests for high-acuity information processing¹. Thus, our reading ability depends on the oculomotor performance, whose properties are reflected most clearly in the statistics of within-word fixations, typically the eyes’ landing position on words after saccades.

Unlike the well-documented effects of cognitive modulation on temporal aspects (e.g. fixation duration) of eye movement measures [e.g.^4,5], small effects of cognitive modulation on spatial aspects (e.g. within-word landing position) of eye movement measures were reported⁶. For example, orthographic familiarity and regularity influence landing positions^7–10. Furthermore, corpus analyses showed a significant effect of word frequency on mean fixation position: Saccades landed further into the (3- to 6-letter) target word, when it was a high-frequency word as compared to a low-frequency word¹¹. A shift of initial fixation location toward the end of high predictable words were reported, but only when the words were seen more frequently and for saccades launched near the word beginning¹². On the other hand, Rayner and colleagues¹³ reported that word predictability had little influence on initial landing position on word and suggested that landing position effects in reading were primarily modulated by low-level processing. Finally, data from z-string scanning (where all letters were replaced by the letter “z” or “Z”) indicated that within-word landing position distributions are very stable and do not critically depend on meaningful content (¹⁴; see also¹⁵). Moreover, most effects of higher-level processing on mean fixation position are small ($<0.5$ character spaces).

Interestingly, the within-word landing position were reported to influence the “higher-level” processes. In several studies, it was reported that (isolated) word recognition time was at the minimum when the eyes land at the center of the word compared to when the eyes land on the word’s periphery, termed the optimal viewing position^16,17. A similar but weaker effect was observed on refixation probability in reading: readers were less likely to refixate the words if fixation land near the word center¹⁸. Oppositely, mean single fixation duration is the greatest when the eyes land at word center, termed the inverted optimal viewing position in Vitu et al.¹⁹. To explain the effect, Nuthmann and colleagues²⁰ argued that fixations landed on the word edges were typically not intended to land on the target (mislocated fixations, see also²¹), reflected on the short duration.

The observed evidence reported in reading research is the result of the integration of cognitive processes in word selection and oculomotor processes in shifting the eyes to the area of interest. To isolate the underlying process affected the observation, one should manipulate factors associated to one process while controlling the remaining factors. Kolers and Perkins²² used geometric rotations, reflections and other transformations of text as the physical variation to study the recognizability of the texts and the influence that practice in reading one type of transformation applied on the recognition of others. They found that the transformations being tested varied in difficulty and transferability. Likewise, Kowler and Anton²³ applied similar types of transformation to test the effects on global saccade lengths and fixation durations. By observing eye movement patterns of two participants, they reported that the directional pattern of saccades had relatively modest effects on reading speed under the instruction to read accurately. They argued that the reading time was affected by longer time needed to generate short saccades observed in reading difficult texts. In a separate test, they found negative relationship between saccade length and saccade latency: Short saccades (less than 30’) have longer latency than long saccades. Additionally, as a response to the Internet myth, Rayner et al.²⁴ tested different types of word transposition (internal, beginning, and end of word) on reading time and reported that although participants were able to read the text, reading time was slower for some transposition types, especially when the word beginning was transposed. Hence they concluded the importance of word beginning (see also²⁵ for similar conclusion). Following the above approach of presenting texts in unfamiliar representation, we designed a study with four different experimental conditions and a control condition to systematically investigate the possible modulations of oculomotor processes in response to variations of the spatial layout of texts. Furthermore, the current study employs various eye movement measures to describe the oculomotor performances during reading.

In this study, letter positions and word representations were experimentally manipulated in the following ways: We used texts composed of mirror-inverted letters (mL), mirror-inverted words (mW), inverted words (iW), where regular letters are printed in reverse order, and scrambled letters (sL) (see Fig. 1a for an example). In mirrored-word (mW) and mirrored-letter (mL) conditions, either the complete word or the constituting letters were mirror-inverted with respect to the vertical axis. In contrast, no mirroring was involved in the construction of inverted-word (iW) and scrambled-word (sL) text conditions. In inverted words (iW) condition, letter representation was normal, but the position was inverted in the iW condition to mimic the letter position in the mW condition. In scrambled letters (sL) condition, the positions of the first and the last letter of a word were maintained, while the letter positions in between were randomized (i.e., there was no change in words with length less than 4 letters). The condition mL, mW and iW are equivalent to the condition NNV, NRV and NRN applied in the study reported by Kowler and Anton²³ while the condition sL is equivalent to the internal transposition manipulation in the study reported by Rayner and colleagues²⁴.

Figure 1.

Experimental sentence stimuli and hypotheses on saccade lengths. (a) In the control condition, normal German text (N) is presented; for mirrored letters (mL) and scrambled words (sL), we expect shorter saccade lengths, on average. For mirrored words (mW) and inverted words (iW), word beginnings and word ends are exchanged, so that we expect longer mean saccade lengths and more regressions due to more frequent re-readings of the same string. (b) The distribution of saccade lengths (solid lines) and the expected changes due to the experimental manipulations.

The following sections describe the robust finding on within-word landing position distribution during reading and the proposed models to explain the observed phenomena. Furthermore, we will discuss about current reading models and their predictions, particularly on saccade generation and “where” the eyes land, in relation to the manipulations in current study.

Within-word landing positions

Regarding “where” the eyes land during reading, a robust finding is that within-word landing position approximately follows a Gaussian density function, with a pronounced peak, typically located halfway between word beginning and word center, but with a surprisingly large variance^26–28. The landmark study by McConkie et al.²⁷ identified two independent oculomotor error components in reading, which we will denote as the range-error model throughout this article: (i) The random placement error is assumed to reflect perceptuo-oculomotor inaccuracy in the execution of saccades, which can be approximated by a Gaussian distribution. (ii) The saccadic range error represents a systematic, launch-site contingent shift of mean landing positions and is typically explained as a general response bias of the human motor system²⁹.

Specifically, McConkie and colleagues²⁷ found that during reading, the within-word landing positions varied systematically with the launch-site distance, i.e., the distance between fixation location (before the saccade) and the beginning of the target word. The stable observation in reading is that each letter increment of the saccade launch-site generates a shift of mean landing-site with a magnitude of about half a character space to the left, the launch-site effect, which is independent of the target word length. If the distances between launch-sites and landing sites are measured relative to word centers, then the within-word mean landing position can be described by a linear landing-position function (see also³⁰) of the form

$${\mathrm{\Delta }}_{\mu }=\lambda \cdot \left(L_0-L\right),$$ 1

where ${\mathrm{\Delta }}_{\mu }$ denotes the average shift of the within-word mean landing position from the word center and $L$ is the distance between launch site and the center of the target word. While a negative value of ${\mathrm{\Delta }}_{\mu }$ indicates that the within-word mean landing position shifts to the left of the word center, a positive value indicates a rightward shift. The parameter $L_0$ represents the center-based launch-site distance, where saccades land precisely on the word center, on average. The strength of the launch-site effect is represented by the slope parameter $\lambda$. An estimated slope of $\lambda \approx 0.5$ was observed in readers of English²⁷ and German²⁰ texts.

While the range-error model was generally a successful first description of the eye-movement data, experimental studies demonstrated effects that could not be explained within this model. First, the presence of additional stimuli could influence the saccadic landing positions^31–36. Second, Kr�ijgel and Engbert³⁷ demonstrated that saccade type (i.e. word skipping) could influence the saccade landing positions during reading [see also^38,39]. These findings challenged the range-error model in explaining observed saccadic landing positions during reading.

It is important to note, however, that the range-error model is purely descriptive, since it does not include more fundamental computational principles for oculomotor control. Furthermore, the slope parameter of $\lambda$ represents quantification of the strength of the launch-site effect, without direct inferences on what processes underlying the observed phenomenon. As a consequence, it is not surprising that integrating new experimental evidence in the range-error model is difficult. In the next section, we discuss a process-oriented Bayesian model of within-word fixation position that was developed over the last 10 years.

A Bayesian model of oculomotor control in reading

The framework of Bayesian decision theory has been proposed as a principled approach to the optimal control of human behavior in the context of integration of sensorimotor and cognitive processes^40–42. Since saccadic eye movements require both sensorimotor (i.e., moving the eyes to foveate words) and cognitive processes, Engbert and Krügel⁴³ proposed that eye movement control during reading could be explained using Bayesian estimation. According to Bayes rule (e.g.⁴⁴), the optimal estimate of a target position $x$, given a sensory observation at $x_0$, can be calculated as the conditional probability (posterior)

$$\pi \left(x\mid x_0\right)~q\left(x_0\mid x\right)p\left(x\right),$$ 2

where $p\left(x\right)$ is the previously learned prior distribution of the target, independent of current sensory input, and the conditional probability $q\left(x_0\mid x\right)$ is the sensory likelihood of the observation at position $x_0$ given a target at $x$. The relation in Eq. (2) determines the dependence of the posterior probability from $x$. The missing constant of proportionality can be obtained by normalization.

Engbert and Krügel³⁷ proposed that the dependence of landing positions within a word on the saccadic launch site is a special case of the Bayesian principles, Eq. (2), for saccade planning in reading. In a mathematical model, the likelihood $q\left(x_0\mid x\right)$ was modeled as an unbiased, normally distributed probability density centered at the intended target word and with variance ${\sigma }_0^2$, which represents the degree of sensory uncertainty. Assuming that the prior distribution is also normally distributed, the product of the two Gaussian density functions results in another normal density function, the posterior probability density $\pi \left(x\mid x_0\right)$. The posterior probability provides a natural explanation of the launch-site effect, since the position of its maximum falls between the maximum of the prior distribution and the maximum of the sensory likelihood. As a result, the posterior reproduces the systematic tendencies of saccades (i) to overshoot the center of close target words and (ii) to undershoot the center of distant target words⁴³.

The shift of the mean of the posterior ${\mu }_P$ from the observation $x_0$ can be calculated as

$${\mathrm{\Delta }}_{Bayes}={\mu }_P-x_0=\frac{{\sigma }_0^2}{{\sigma }_0^2+{\sigma }_T^2}\left({\mu }_T-x_0\right),$$ 3

where ${\sigma }_T^2$ is the variance of the prior probability of center-based launch-site distances. Comparing the equation for the launch-site effect, Eq. (1), with the predicted effect in the Bayesian theory, Eq. (3), thus assuming ${\mathrm{\Delta }}_{\mu }\equiv {\mathrm{\Delta }}_{Bayes}$,we obtain

$$\lambda =\frac{{\sigma }_0^2}{{\sigma }_0^2+{\sigma }_T^2}.$$ 4

Recently, Krügel and Engbert⁴⁵ introduced an advanced model, which includes an explicit model for the computation of the word center from sensory estimates of word boundaries. Therefore, Bayesian models provide a robust theoretical framework to explain where the eyes move during reading. One advantage to model within-word landing position distribution using an explicit Bayesian model is that the interpretability of the results. The slope parameter $\lambda$ estimated from Bayesian model represents weighting of optimal behavior during reading: maintaining constant saccade length while targeting word center. For extreme case where $\lambda \to 0$, it can be interpreted that the optimal oculomotor control in reading puts more weight on the precision in landing on target precision, hence minimizing range error. On the other hand, the value of $\lambda \to 1$ means that optimal reading behavior rely on maintaining constant saccade length, reducing the importance of target location. With the typical empirical value of $\lambda \approx 0.5$, it can be interpreted that the sensory variance of the target location is approximately the same as the variance of the prior distribution, i.e., ${\sigma }_0^2\approx {\sigma }_T^2$, in optimal reading behavior.

Hypothesis and predictions from various reading models

Reading models were typically developed to help understanding the complex processes underlying reading processes. Despite the fact that eye movement control during reading requires both cognition and oculomotor systems, as mentioned above, empirical studies found that cognition had small effects on within-word landing positions. Interestingly, adding additional visual cue¹¹ or changing sentence presentation, i.e. texts are read from top to bottom⁴⁶ did not notably change the landing position distributions on words. Consequently, most mathematical models of saccade generation during reading assume that oculomotor control is dissociated from cognitive processes. Cognitive-based reading models (e.g., E-Z Reader^47–49; SWIFT^2,50,51) assume that cognitive processes related to language processing are responsible for eye movements without distinguished effects on oculomotor within-word targeting process. Moreover, most cognitive models based their implementation of saccadic errors on the range-error model²⁷ with relatively fixed values. Since the manipulation types tested in current study maintain the spatial information such as word length, cognitive-based models will not predict substantial changes in within-word landing positions since they assume that oculomotor process is mainly affected by “low level” information. However, since text manipulations will increase processing loads, hence affecting saccade generating time, these models will generate more refixation saccades but less skipping saccades.

Most relevant to current study is Mr. Chips^52,53, an ideal-observer model that combine visual, lexical, and oculomotor information optimally to read simple texts in the minimum number of saccades. The model operates according to an entropy-minimization principle, generating saccades that minimize uncertainty about the current word or saccades that move the visual span furthest to the right. Note that for Mr.Chips, a word is said to be fixated if the central slot of the visual span (a linear array of character slots with each slot can be either high or low resolution) falls on one of the letters of the word and this central slot does not have preferred status. Mr. Chips’ skipping rate and global landing position distribution were similar to human data. Mean saccade length decreases as the lexicon size increases. Furthermore, it generated less refixations and reduced launch-site effect ($\lambda =0.21$) in reading normal text. Given that word identification played a key role for lexical processing in the model, we speculate that if the words were written from right to left (e.g. in mW and iW conditions), Mr. Chips should generate more saccades that land on the second half of a word to capture more information about word identity, assuming that the first half of the word were identified beforehand and letter mirroring and inversion do not affect it’s lexical access process.

Our hypothesis for eye-movement measures on reading unfamiliar typography are derived from the predicted increase in perceptual difficulty and additional oculomotor demand. Longer fixation durations and shorter saccade amplitudes can be expected for more difficult texts in all four conditions. Increased perceptual difficulty should also result in less skipping cases, but more refixations; both of these predictions are compatible with a reduced average saccade length (see Fig. 1b).

On the level of within-word landing positions, our hypotheses are more specific for the different experimental conditions. Reading words with mirror-inverted letters (mL) or with scrambled letters (sL) will produce within-word landing positions similar to normal reading, since information on letter positions did not change (mL) or did not deviate systematically from normal reading (sL). However, due to inversion of letter positions in reading mirrored-words (mW) and inverted words (iW), we expect readers to shift their eyes further to the right of the word string in the initial saccade and to generate a regressive refixation after that initial saccade (see Fig. 1a).

Our study employs a learning paradigm, where participants are required to train reading text in one of the four experimental conditions. The motivation for the learning paradigm is to check the stability of the resulting eye-movement patterns and to exclude the possibility that the results are a short-lived effect due to first exposure to an unfamiliar layout.

In general, we expect that the training will lead to improved performance on the level of global reading measures (average fixation times and mean saccade lengths). Since the scrambled letters (sL) condition represents the least systematic variation of the text layout, we expect that the learning effects are smaller than in the other three experimental conditions. On the level of within-word landing positions, we expect that stable shift of the initial landing positions toward the end of the word strings will be established during learning in the conditions with mirrored words (mW) and inverted words (iW), since the word beginnings are at the end of the manipulated string in these two conditions. For the other two conditions (mL, sL), we expect that within-word landing positions are very similar to normal reading after training.

Methods

Participants

A group of 37 participants (27 females, 10 males), aged between 16 and 39 years, received a total of EUR 70 for taking part in four 45-minute lab sessions and six 30-minute training sessions. They were all naive with respect to the purpose of the experiment. Participants reported normal or corrected-to-normal vision and declared their informed consent. The experiment conformed to the Declaration of Helsinki. Informed consent was obtained for experimentation by all participants. According to the standards of Deutsche Forschungsgemeinschaft (German Research Foundation) and German Society for Psychological Research, ethics committee approval was not required for this study.

Apparatus, material & procedure

Participants were assigned to four different groups based on four types of text manipulation, namely mirrored-word text (mW), mirrored-letter text (mL), inverted-word text (iW), and scrambled-word text (sL). Each participant did a total of four lab sessions and six training sessions at home via web-based interface.

For lab sessions, participants read an excerpt from the German version of the novel “The Adventure of the Empty House”⁵⁴. They were seated at a viewing distance of 70cm in front of a 19-inch Mitsubishi Diamond Pro 2070 Monitor (screen resolution 1,280 $\times$ 1,024 pixels, refresh rate 100 Hz) with the head supported by a chin rest. The stimuli (Courier font, size 18, black) were presented on vertical center line of the computer display with gray background color. Eye movements from both eyes were recorded using an EyeLink 1000 System (SR Research, Osgoode/Ontario, Canada) with a sampling rate of 1000 Hz and spatial resolution better than 0.01${}^{\circ }$. At the end of the session, participants had to answer three questions related to the text they have just read. A maximum of 600 lines of text were presented over all lab sessions.

For training sessions, participants read an excerpt from the novel “Small World”⁵⁵ by visiting a website created for the experiment (using ShinyApps by RStudio accessible via http://www.shinyapps.io). After logging in, participants could read the manipulated text, which was presented as a line of max. 85 characters at the center of the screen, without time limit. When finished, they could go to the next line by clicking the right arrow or return to the previous page by clicking the left arrow. One training session should last at least 30 minutes. After logging out, participants received three questions via E-mail, which should be answered as soon as possible. No limit of the number of lines presented in training sessions was enforced.

The complete procedures of the experiment went as follows: During the first lab session, participants read normal text (a total of 150 lines of maximum 85 characters). After a two-hour break, participants read manipulated text in the second session, which lasted up to 45 minutes or when 150 lines were read. At home, participants were required to read manipulated text for two 30-minute sessions on the website before the third lab session. At the third lab session, participants continue reading manipulated text from where they left off at the previous session. Participants should conduct four 30-minute training sessions before taking part in the last lab session.

Data preparation

Data containing blinks were discarded from the analysis. Saccades and fixations were detected using a velocity-based algorithm developed by Engbert et al.⁵⁶. As a result, a total of 380,292 fixations were detected. From this data set we excluded fixations based on the following criteria (i) fixations on the first and last words of a sentence as well as the first and the last of participant’s fixation sequence, (ii) fixations with duration less than 20 ms or longer than 1000 ms, fixations landing outside the text rectangle, and saccades shorter than one character space (12 pixels) or longer than 25 character spaces were removed from the analysis. Trials containing fixation duration longer than 2000 ms and less than three fixation points after filtering were excluded from analyis. The remaining 236,937 fixations are the valid fixations (see Table A1 in Supplementary Information).

Results

Reading text with manipulated layout due to mirrored-reversed and inverted letter arrangement produced changes in behavioral measures on a global level (e.g., mean fixation duration, average saccade length) and on the oculomotor level (i.e., within-word fixation locations/saccadic landing positions). While the primary goal of our study was to investigate the possible modulations of oculomotor processes in response to variations of the spatial layout of texts, we start with the presentation of results on global summary statistics to evaluate the overall effects on reading performance as characterized by eye-movement measures.

Global summary statistics

This section highlights the significant results for each manipulation across training session in comparison with normal reading. If not otherwise mentioned, results refer to comparisons between the first experimental session (in one of the four conditions mW, iW, mL, or sL) and the control condition (normal reading).

First of all, global summary statistics from reading normal texts are replicated. In control condition, most of the first-pass fixations move forward to the next neighbouring words (forward saccades 41%) or skip (skipping saccades 26%). About 22% of the fixations move within a word (refixation saccades) and 12% of them move backward (regression saccades). On average, readers move their eyes 7.82 character spaces forward, 4.23 character spaces backward and fixated on words for 245 ms during reading normal text. Compared to control condition, percentage of forward saccades in experimental condition did not show strong deviation (31–44%). The percentage of skipping saccades was reduced to less than 5% in conditions where texts were written against the reading direction (e.g. in mW and iW conditions) and between 10–15% in other conditions. Interestingly, half of the first-pass fixations observed in conditions where texts were written against reading direction came from refixation saccades, while only a third of first-pass fixations were generated from refixation saccades. Less than 12% of fixations in reading manipulated text were moving backwards (e.g. regression saccades) with the lowest observed in iW condition. When reading the manipulated texts for the first time, readers fixated on words, on average, about 53–137 ms longer and generated shorter saccades (5–6 character spaces for progressive saccades and 3 character spaces) than those observed in reading normal text. Note that mean fixation duration in sL condition is about 50 ms longer than those reported in the study by White and colleagues²⁵. Global summary statistics from all sessions are presented in Table A1 in Supplementary Information.

On a qualitative level, we compare the resulting saccade length distributions (Fig. 2) with our hypotheses (Fig. 1b). As expected, the distributions of forward saccades length during reading manipulated text are generally shifted to the left of those from control condition, indicating a qualitatively shorter saccade length generated. Futhermore, more regressive saccades were observed only in reading mirrored-word (mW) and inverted-word (iW) conditions.

Figure 2.

Distribution of saccade amplitudes across all sessions. Saccades observed in the control condition (normal reading) are marked with red color. Dark blue lines represent data from first session of reading manipulated text. Data from the last two training sessions are marked with lighter blue hues.

Single fixation duration and refixation probability

Results from statistical modeling using linear mixed-effects models for single fixation duration and refixation probability are summarized in Tables A2–A4. For specification of linear mixed-effects models using the lme4 package⁵⁷, see Section 1.1 in Supplementary Information.

No significant word-length effect was observed in single fixation duration measure for control condition, which is in line with the finding reported in study conducted by Kliegl et al.⁴. Compared to control condition, word-based single fixation duration observed in experimental conditions is significantly longer (all $t$ value $>$ 2). For example, on medium words (5–7 characters), estimated mean single fixation duration in the first experimental sessions were 48–104 ms longer than in control condition (estimated mean: 238 ms). In line with the study by Kowler and Anton²³, texts written against reading direction were more difficult to process. This processing difficulties seemed to be the property of the corresponding type of text manipulation and were more obvious after participants have learned how to read the texts. When readers fixated on medium-length words only once in the last experimental sessions, the fixation duration in mL (estimated mean: 309 ms) and and sL (estimated mean: 326 ms) conditions were shorter compared to those observed in mW and iW conditions with estimated mean of 570 ms and 455 ms respectively.

Similar to single fixation measures, no significant word-length effect was observed in first of multiple fixation duration measures for the control condition. When words were fixated more than once, the duration of the first fixation in the sL condition (across all sessions) was significantly longer than in the control condition. In the mL condition, the duration of the first fixation was significantly longer than in the normal condition only in the first experimental session. After training, no significant difference was observed. Interestingly, in comparison to the control condition, the mean of first fixation duration was significantly longer in mW and iW conditions. For example, on medium-length words, estimated mean duration of the first fixation in both conditions were about 90 ms longer than those estimated for control condition. The difference remained significant even after training (estimated means session 3: $mW=465.32$; $iW=414.38$).

On refixation probability measure, we replicated the word-length effect in both control and experimental conditions (all $p<0.01$). Long words were more likely to be refixated than short words. However, when the texts were manipulated, readers were more likely to refixate the words. For example, the estimated refixation probability on medium words in control condition was 0.16. However, readers in experimental conditions were twice more likely to re-fixate the words with the same lengths (estimated RFP: $mL=0.42$; $sL=0.65$; $mW=0.44$; $iW=0.56$). Interestingly, after training to read texts with words written against the reading direction (e.g., mW and iW conditions), readers almost always fixated on medium and long words more than once (estimated RFP above 0.87 on the third session).

Do these manipulation types have something in common or do they generate different effect on fixation duration and probability measures presented above? The results from separate models which estimated the different effect size of manipulation types (see Table A5 in Supplementary Information) confirmed that some manipulation type generated greater effect than the others. All of the three models showed similar trends. The effect in mL condition was significantly different from control condition, but adding the effect from sL condition to the mean of the two conditions (control and mL conditions) did not yield a significant gain on effect size. However, adding the effect from mW condition and iW conditions yielded significant gains on the effect size. The results from various measures showed that of all manipulation types tested in the current study, the iW condition generated the largest effect.

The effects of word lengths on duration of single fixation and first of multiple fixations, as well as on skipping and refixation probabilities are visualized in Fig. 3.

Figure 3.

Upper row. Mean fixation duration of single fixation (left) and first of multiple fixations as the function of word length classes. Lower row. Skipping (left) and refixation probabilities as the function of word length classes. Word with length less than 3 characters are grouped in word length class 3; long words ($>8$ characters) are grouped in word length class 8. Error bars represent the standard error of the means. Red line and dots represent data from reading normal text. The dark blue color represents data from the first experimental session. The lighter blue hues indicate data from the last two experimental sessions.

Landing positions distributions and launch-site effect

A first glance at the resulting distributions for within-word landing positions indicates differences between normal reading and manipulated texts (Figs. F1–F4 in Supplementary Information). Except for short words (i.e., word length up to 4 chars), we found increased leftward shifts across all experimental conditions. Stronger shifts in the mirrored-word (mW) and inverted-word (iW) conditions were observed, in comparison to the other two conditions (mL, sL). Since saccades observed in the experimental conditions were shorter than those observed in the control condition, we analyzed only saccades that were launched up to 5 characters away from the word beginning. As a result, more overshoots were reported in this study. Undershoots were typically observed on longer saccades launched further than 7 characters away and more prominent on long words^20,27,37. Based on Bayesian fits58,59 of the distribution (see Supplementary Information), we obtained mean landing positions that were used for further analyses.

In Fig. 4a, center-based mean landing sites are plotted against center-based launch sites for forward saccades and different lengths of the target words. The slope parameters from landing position functions are plotted for all sessions in Fig. 4b. The estimated slope of (${\lambda }_0=0.37$) was observed in reading normal text. Compared to the value from control condition, both conditions iW (${\lambda }_1=0.31$, ${\lambda }_2=0.25$, ${\lambda }_3=0.23$) and mW (${\lambda }_1=0.24$, ${\lambda }_2=0.26$, ${\lambda }_3=0.29$) generated shallow slopes, which indicates a tendency to reduced oculomotor control. In contrast, conditions mL (${\lambda }_1=0.44$, ${\lambda }_2=0.38$, $\lambda =0.37$) and sL (${\lambda }_1=0.52$, ${\lambda }_2=0.48$, $\lambda =0.41$) generated greater slope values ($\lambda$) compared to reading normal text, indicating a tendency to increased oculomotor control. Interestingly, except in the mW condition, the $\lambda$ values approach the value of normal reading after trainings.

Figure 4.

Analysis of the launch-site effect on the systematic error component. (a) Landing position function: Mean center-based landing position as a function of center-based launch-site distance across all sessions. Red lines and dots represent data from normal reading session. Data from the first experimental sessions are presented in dark blue color. The lighter blue hues represent the last two experimental sessions. (b) The estimated slope parameter $\lambda$ (and the 95% confidence interval) over the course of training in the experimental conditions. The red horizontal dashed line represents the estimated slope for reading normal text (baseline). Results for reading mirrored words (mW) and inverted words (iW) are presented with magenta and cyan lines, resp. Green and yellow lines represent estimations for reading texts with mirrored letters (mL) and scrambled letters (sL).

In the Bayesian model, the slope parameter $\lambda$ represents the strength of oculomotor control and the relation between observational error of the target location (${\sigma }_0^2$) and standard deviation of prior distribution (${\sigma }_T^2$). In extreme cases, a value of slope near to 0, where ${\sigma }_0^2$$\to$0, the eyes always land on the target (i.e., the word center) regardless of where the saccade started. In the other extreme case, the a slope value of $\lambda \to 1$ indicates the absence of a target selection process in saccade planning, so that the eyes generate random constant saccade lengths (from a uniform distribution). Therefore, our data show that saccades from reading text composed of words with reversed letter sequences (i.e., mW and iW) tend to land precisely on the target location (word center) on average, while readers of manipulated texts with normal letter order (i.e., mL and sL) tend to generate saccades with similar lengths.

Comparing the results with our hypotheses, there is no dramatic effect of a shift of the mean landing position toward the word ends. Therefore, we ran a post-hoc analyses for single and two-fixation cases in the next section.

Furthermore, we also considered saccadic precision in word-to-word forward saccades. Figure 5 shows the standard deviations of landing positions in normal reading and in reading with manipulated text as a function of launch-site distance. The main effect on standard deviation (${\sigma }_{SF}$) of within-word landing positions of fixations coming from word-to-word forward saccades yielded an $F$-ratio of $F\left[1,198\right]=33.8$, $p<0.000$, indicating a significant difference between reading normal text (M = 1.85, SD = 0.30) and manipulated text (M = 1.54, SD = 0.36). Most importantly, it turned out that the variance of landing-position distributions in all experimental conditions with manipulated texts are lower compared to normal reading. Thus, reading under manipulated text conditions led to an increased overall precision of readers’ saccades.

Figure 5.

The launch-site effect on the standard deviations of the distributions of within-word positions for fixations coming from word-to-word forward saccades. Red lines and dots represent data from normal reading session. Data from the first experimental sessions are presented in dark blue color. The lighter blue hues represent the last two experimental sessions.

Landing positions for single vs. two-fixation cases

The manipulation of text displays resulted in the changes of the proportion of cases where words were fixated only once, twice, or multiple times (see Table 1 for summary). Under normal reading condition, most words were fixated only once (85%); some words were fixated exactly twice (14%) and a few words received multiple fixations (2%). When readers were first confronted with manipulated texts, however, the observed single fixation cases were reduced to about 58% to 74%. About 20% of two-fixation cases and 12% of multiple-fixation cases were observed in the first non-normal reading sessions.

Table 1.

Relative frequency of single, two-fixation and multiple-fixation cases [in %].

Condition	Session	Single fixation cases	Two-fixation cases	Multiple fixation cases
Control	0	84.21	13.82	1.97
Mirrored-Word (mW)	1	63.29	20.4	16.31
	2	66.39	19.66	13.95
	3	67.72	18.39	13.89
Mirrored-Letter (mL)	1	71.41	21.04	7.56
	2	74.97	20.39	4.64
	3	78.56	17.33	4.11
Inverted-Word (iW)	1	58.18	21.99	19.83
	2	61.98	21.97	16.05
	3	61.54	22.1	16.36
Scrambled-Letter (sL)	1	74.13	18.24	7.63
	2	74.3	17.82	7.87
	3	72.87	19.3	7.83

Single fixation cases

Only cases where exactly one fixation land on a word were considered for this analysis. Relative frequency of fixation landing position were calculated based on word length. To obtain estimates for the mean ${\mu }_{SF}$ and standard deviation ${\sigma }_{SF}$ for the landing position distribution of each word length, a grid search method (in steps of 0.01) with a minimum-${\chi }^2$ criterion was applied. Analysis of variance was conducted to statistically compare the manipulation effects on mean (${\mu }_{SF}$) and standard deviation (${\sigma }_{SF}$) of landing position distributions.

No significant difference was observed for the mean (${\mu }_{SF}$) of the landing position distribution of single fixation cases between control and experimental conditions ($F\left[1,30\right]=2.364$, $p=0.135$). The main effect on standard deviation (${\sigma }_{SF}$) of within-word landing position distribution of single fixation cases yielded an $F$-ratio of $F\left[1,30\right]=32.7$, $p<0.000$, indicating a significant difference between reading normal text (M = 2.95, SD = 0.84) and manipulated text (M = 1.51, SD = 0.55). When words were fixated only once, readers’ mean landing position did not change across the manipulation types. However, the precision of landing on selected target increased as the text display deviated from normal presentation.

Among our key results is that the two conditions where the word beginnings were located at the end of the manipulated word strings (i.e., conditions mW and iW) produced only a slight shift of the average landing position. This finding turned out to be robust and remained observable even after trainings. Figure 6 presents data for single fixations. The finding did not support our hypothesis that readers targeted the second half of the word strings during reading mirrored-word (mW) and inverted words (iW). On a qualitative level, there is little adaptivity of the oculomotor system.

Figure 6.

Within-word landing position distributions for single-fixation cases. Columns relate to different word lengths. Rows refer to experimental manipulations. Different colors indicate control condition (red) and experimental conditions (blue hues).

Since in our hypothesis, an initial saccade toward the second half of the word would require an additional refixation in the first half of the word, we ran a post-hoc analysis of all two-fixation cases. The corresponding plot in Fig. 7 presents data for the initial landing position (first saccade into the word) from all cases, where the word was fixated exactly two times. In contrast to typical OVP effect on refixation probability, plotting the initial of two fixations give us a better understanding of word targeting in saccade planning.

Figure 7.

Within-word landing position distribution for the initial landing position (first saccade) in two-fixation cases. Red dots and curves represent the fixations observed in control condition. Blue hues represent the fixations observed in experimental conditions.

Two-fixation cases

For this analysis, we considered cases when exactly two fixations on a word were observed. The landing position distribution of the initial fixation was fitted to a quadratic polynomial, i.e.,

$$y=A+B\cdot L{\left(x-C\right)}^2,$$ 5

where $x$ denotes the initial landing position and y is the relative frequency of the fixation. The parameter A represents the actual relative frequency of the initial landing position. The parameter B is the slope of the parabolic curve, representing the within-word maximum or minimum relative frequency of the landing position. The parameter C reflects within-word position where the relative frequency was at the minimum or maximum, depending on the value of parameter B. In general, the value of the parameter B is assumed to be positive, resulting in a distribution of landing position qualitatively similar to the optimal viewing position (OVP) curve but with different interpretation: when a word is fixated exactly twice, the initial fixations tend to be near word edges more often than on the word center. The estimation of the three parameters representing the characteristics of the landing position of the first of two fixation cases was conducted based on a maximum likelihood method using the bbmle package⁶⁰ for R studio. The estimated parameters are summarized in Tables A6–A10.

Under normal reading conditions, refixational saccades are mainly launched from initial fixations near the word beginning with a subsequent secondary fixation further into the word (most refixations are directed toward reading direction). Similar patterns were observed in the condition where manipulated texts were written in typical direction from left to right (mL and sL). However, in the two conditions (mW and iW) with reversed letter sequences, the initial landing positions deviated from those of normal reading (for detailed estimated parameters, see Tables A6–A10 in Supplementary Information). When a word was fixated more than once, the first saccade landed mostly behind the location of the word center in the second half of the word strings. The effect was stronger for longer words (see Fig. 7). This finding matches our initial hypothesis that readers direct the eyes at first into the second half of the reversed words and then generate a secondary within-word saccades along writing direction. However, as we found no support of an overall shift of all initial saccades for the conditions mW and iW in our previous analyses, a possible explanation of the observed shift in the initial landing positions of two-fixation cases is a strategic modulation of readers’ within-word targeting process.

Discussion

Natural text reading requires efficient coordination of cognitive and oculomotor control processes. Cognitive principles are essential for the selection of an upcoming target word, however, it is the oculomotor system that provides the machinery to move the gaze to the region of the identified target word. In the current study, we set out to investigate whether ongoing cognition is able to overwrite default oculomotor control when the reader is confronted with manipulated (mirrored-reversed, inverted, and scrambled) text layouts.

On a global level, changing the display and positions of letters in a word modulates eye-movement statistics. We observed longer average fixation duration and shorter mean saccade amplitude in reading conditions with non-normal text layout. These findings support the conclusion from previous studies^23,24 that changes on the level of letters lead to increased costs for word recognition in comparison to reading normal text. Interestingly, similar with findings from previous studies^22,23, orthographical manipulations from our current study can be grouped into two categories based on their writing directions. Difficulties increased dramatically, when readers read texts written against the reading direction as in mirrored words (mW) and inverted words (iW) conditions, where the letter-sequence of words is written from right to left within the displayed string due to mirroring or inversion. Our interpretation is supported by the fact that the observed deviations remained stable even after training. In contrast, during the manipulations that kept the letter-sequence in the reading direction (i.e., from left to right) in mirrored-letter word (mL), the patterns observed in eye movement measures (e.g., word-length effect) remained similar to those of normal reading and, after training, approached the behavior observed in normal reading. Finally, in our fourth experimental condition, we analyzed reading text composed of scrambled-letter (sL) words. Since letters in a word were randomly scrambled, there was no systematic way that the oculomotor behavior could adapt to process the new text layout.

One of our motivations to run the current study was to obtain a detailed picture on within-word landing positions as the most important signature for oculomotor control. Mean landing position distribution of single fixations in reading manipulated texts does not significantly differ from reading normal text, although there was a slight shift observed in mirrored words (mW) and inverted words (iW) conditions. However, readers tend to increase their precision in landing on word center when texts presentation was manipulated. Increased saccadic precision was found for word-to-word forwards saccades in all experimental conditions compared to normal reading and the result also holds, if only single-fixation cases were considered. The finding supports the idea that word center serves as saccadic target location⁴³. Analysis of mean landing position of forward saccades based on different launch sites demonstrated a general left shifts in experimental conditions compared to those observed in control condition. Especially in conditions where words are written against reading direction such as in mW and iW conditions, there is no clear preference for the eyes to initially land in the second half of the word strings (where the beginning of a word is located in this manipulation). This is contrary to our hypothesis. Interestingly, the peaks of the landing position in all manipulation types were shifted toward word centers, not necessarily toward the beginning of word strings, compared to the peaks observed in reading normal text. In some rare undershoot cases in our data (e.g. launch site: $-4$, word length: 7 in iW and mW conditions), the peaks in experimental condition shift rightward of the peak observed in reading normal text. Furthermore, the variance of the initial landing position distribution was much smaller, meaning that the eyes landed more precisely on the word center, which served as the theoretical target position. Given the precision of the landing position distributions and the highest likelihood to land on word center, we speculate that the leftward shifts were actually moving toward the word centers, not word beginning. Regarding the general improved precision of landing position distributions, a possible explanation is that with longer average fixation durations under our manipulations, the oculomotor system had more time to prepare the next saccade, which could result in a reduced saccadic error²⁷. An alternative explanation is that unusual presentation of text was more salient and popped up parafoveally to enable precise saccadic targeting⁷.

Additionally, modulation of manipulation types on launch-site effect was observed. In conditions where texts were written against reading direction (from right to left), we observed a reduced launch-site effect, while an increased launch-site effect was observed in conditions where texts were written normally. However, we are not sure if the difference in the effect size is caused by the change in reading direction or increased processing difficulties. The findings could be explained in the theoretical framework of a Bayesian model of sensorimotor integration that we applied to the oculomotor control during reading^43,45. According to Bayes’ rule, the observed landing position distribution is the posterior distribution in a sensorimotor transform based on prior knowledge of typical target positions. If we assume that the prior knowledge is uninformative under unfamiliar text layout, then the posterior distribution would shift toward the likelihood distribution, which is assumed to be unbiased with respect to the target word center. Therefore, in the Bayesian model, we could have expected a more centered landing-position distribution if we assumed that the experimental manipulations induce less prior knowledge on the possible target positions. Our finding demonstrated that by simply changing the letter-sequence information of words, we could obtain estimations for the slope parameter $\lambda$ other than the typical value of 0.5. As a consequence, reading models development should aim for integrating a process-oriented model in generating saccade lengths.

The most striking finding in the current study is the effect of orthographic manipulation on the initial landing position of two-fixation cases. When a word shows reversed letter ordering (hence a change in their spatial information) as a consequence of the experimental manipulation, we observed a clear shift into the second half of the word string when two fixations were generated. This finding suggests that the oculomotor system is able to adapt to display changes. Given that our hypothesis requires more than two fixations, analysing the initial landing positions in refixation cases is more reasonable to test the hypothesis.

Furthermore, our study also demonstrates that the typical usage of mean fixation position as a dependent variable to characterize oculomotor control is not specific enough to characterize control processes underlying reading. Since reading is a complex process, data delivered are usually complicated and require various analysis methods and inferences. Further systematic analyses and advanced mathematical modeling are required to investigate the dynamical processes underlying principles of oculomotor control and their interaction with ongoing cognition during reading.

Author contributions

R.E. and A.K. developed the study concept and the study design. Testing and data collection were performed by J.C. J.C. and A.K. performed the data analysis and all authors contributed to the interpretation. J.C. and A.K. drafted the manuscript, and R.E. provided critical revisions. All authors approved the final version of the manuscript for submission.

Data availability

Data and source code are available via Open Science Foundation at https://osf.io/bmvrx/.

Competing interests

The authors declare no competing interests.

Supplementary information

is available for this paper at 10.1038/s41598-020-60833-6.