Skip to main content
NCBI home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2021 Jul 25.
Published in final edited form as: Ann Dyslexia. 2020 Jul 25;70(2):243–258. doi: 10.1007/s11881-020-00202-0

Using an Item-Specific Predictor to Test the Dimensionality of the Orthographic Choice Task

Donald L Compton 1, Jennifer K Gilbert 2, Devin M Kearns 3, Richard K Olson 4
PMCID: PMC7428840  NIHMSID: NIHMS1615101  PMID: 32712817

Abstract

The orthographic choice (OC) task — requiring individuals to choose the correct spelling between a word and a pseudohomophone foil (e.g., goat vs. gote) — has been used as an outcome measure of orthographic learning and as a predictor of individual differences in word reading development. Some consider the OC task a measure of orthographic knowledge (e.g., Conrad, Harris, & Williams, 2013), whereas others have suggested that the task measures a reader’s familiarity with the word’s orthographic representation and thus measures word reading skill (e.g., Castles & Nation, 2006). We examined this assertion by testing OC task performance of individuals ages 8 to 18 (J = 296) and their ability to read the OC target words (I = 80) in isolation using crossed random effects item-response models. Results reveal that response on the OC task is not fully determined by the ability of an individual to read the target word in isolation. Specifically, the probability of choosing the correct orthographic form when the word was pronounced incorrectly was .79; whereas it was .90 when the word was pronounced correctly. Measures of receptive spelling and phonemic awareness (person-characteristics) and word frequency and orthographic neighborhood size (item-characteristics) accounted for significant variance in orthographic choice after controlling for target item reading and other reading related abilities. We interpret the results to suggest that the OC task taps both item-specific orthographic knowledge and more general orthographic knowledge.

There is near unanimity that phonological processing ability is central to the acquisition of early word recognition skills (see Adams, 1990; Blachman, 2000; NICHD, 2000; Stanovich, 1986). In addition to phonological processing, orthographic knowledge1 has been identified as an important source of variance in the word recognition performance of children (Barker, Torgesen, & Wagner, 1992; Cunningham & Stanovich, 1990; Manis, Doi, & Bhadha, 2000; Olson, Wise, Conners, Rack, & Fulker, 1989). Generally, orthographic knowledge refers to an understanding of the print conventions used in a writing system as well as knowledge of how words are spelled (Conrad, Harris, & Williams, 2013). Stanovich and West (1989) define orthographic knowledge as involving the “ability to form, store, and access orthographic representations” (p. 404). Measures representing orthographic knowledge have consistently been demonstrated to account for significant variance in word recognition performance in developing readers after the variance associated with phonological processing has been removed (e.g., Conrad et al., 2013; Connors & Olson, 1990; Cunningham & Stanovich, 1990; Cunningham, Perry, & Stanovich, 2001; Deacon, 2012). In addition, orthographic knowledge measures have been used as an outcome measure in various self-teaching experiments (e.g., Cunningham, Perry, Stanovich & Share, 2002; Nation, Angells, & Castles, 2007; Share, 2004; Wang, Nickels, Nation, & Castles, 2013).

The Orthographic Choice Task

The orthographic choice (OC) task (Olson, Forsberg, Wise, & Rack, 1994 Olson et al., 1989) has been used extensively to study the association between orthographic knowledge and word reading development (see Olson, Forsberg, & Wise, 1994). In the OC task, individuals are asked to choose the correct spelling of a target word when presented the word (e.g., take) and a pseudohomophone foil (e.g., taik). The task is designed to allow phonological decoding to occur without the output of this phonological process being sufficient to make a decision about the lexical identity of the letter string. Thus, it is assumed that alphabetic decoding skills alone are insufficient to perform the task and therefore children must rely on orthographic knowledge to correctly select the word.

The OC task has been criticized extensively for tapping word reading skills as opposed to orthographic knowledge (e.g., Burt, 2006; Castles and Nation, 2006; Vellutino et al., 1994). As Castles and Nation (2017) put it, “Measures of orthographic processing skill are difficult if not impossible to distinguish from tests of word recognition” (p. 152). Indeed, orthographic choice measures have been reported to correlate quite highly (ranging from .50 to .70) with measures of word recognition (Conrad, Harris, & Williams, 2013; Cunningham, Stanovich, & West, 1994; Manis et al., 1993; Olson et al., 1989; Stanovich, West, & Cunningham, 1991), suggesting they are tapping the same construct. In a longitudinal study of early reading development, Deacon, Benere and Castles (2012) reported that word reading ability predicted progress in orthographic knowledge acquisition, whereas orthographic knowledge did not predict future word reading skill growth. Results are consistent with a developmental model of reading development in which decoding a word provides children with an opportunity to learn word-specific information about the word, referred to as orthographic learning (Nation & Castles, 2007). Through the process of orthographic learning, the accumulation of word-specific information supplies children with a means to gradually accumulate knowledge about general orthographic relations in their language.

We certainly acknowledge and agree with the argument that the OC task relies heavily on stored word-specific orthographic representations resulting from orthographic learning; however, we wonder whether other more generalized orthographic knowledge (e.g., subword representations) stored during word reading development might affect performance on the OC task. Consistent with this argument, Conrad et al. (2013) reported orthographic knowledge to be a multi-dimensional construct, consisting of both word specific and general orthographic knowledge. Conrad et al. defined general orthographic knowledge as “an awareness of the general attributes of the writing system, including sequential dependencies, structural redundancies, and letter position frequencies” (p. 1225). They reported that both types of orthographic knowledge made separate and unique contributions to both reading and spelling skill, over and above the contributions of phonological skills, in developing readers.

In allowing multiple forms of orthographic knowledge to influence OC performance we adopt an item-level as opposed to a measure-level perspective. In taking an item-level approach we acknowledge the potential influence of two orthographic knowledge sources, namely word-specific lexical and general (Conrad et al., 2013; Deacon, 2021), that can vary by item- and person-characteristics. When a reader has a complete representation of a word then the OC task likely assesses word-specific orthographic knowledge (i.e., word reading skill). However, when the reader has an incomplete orthographic representation of the word the task allows general orthographic knowledge to influence performance. Specifically, in the case of an incomplete representation the reader can rely on (a) incomplete orthographic knowledge about the word, (b) knowledge about sublexical orthographic features of the language, and/or (c) other unspecified orthographic knowledge to correctly distinguish the target word from the pseudohomophone foil. This hypothesis is certainly in keeping with the hypothesis that development of sensitivity to orthographic constraints developed through orthographic learning may assist children in making orthographic choices even when they cannot read the words.

Current Study

The purpose of this study was to test the hypothesis that item performance on the OC task relies on multiple forms of orthographic knowledge that depend on the availability of a word-specific representation in the individual. Our study differs from Conrad et al. (2013) who predicted word reading and spelling skill using measures of OC (e.g., rain-rane) and word-likeness (e.g., vage-vayj), controlling for phonological skill. Instead, we examined whether item-level performance on the OC task is determined by the ability to recognize the target words on the task in isolation and further explored person- and item-characteristics that explain variance in orthographic choice. Because it is impossible to observe the use of different forms of orthographic knowledge during orthographic choice performance, we relied on inferences based on item-level performance on target word reading in isolation and during orthographic choice to separate word-specific from general orthographic knowledge.2 Item-level models provide a unique opportunity to examine the role of item-specific predictors in the presence of more general person and word predictors. It would be nearly impossible to examine the questions posed in the current study without these models.

Three research questions were evaluated in the study. First, we asked whether item-specific word recognition accuracy would significantly explain variance in item-level performance on the OC task? If item-specific word recognition is highly aligned with OC task performance (i.e., little influence of general orthographic knowledge), results would take the following form: (a) for words read correctly in isolation orthographic choice performance should approach 100% (presumably due to word-specific orthographic knowledge) and (b) for words read incorrectly in isolation OC performance should approach 50% (assuming measurement is error free). If however, OC task performance is accomplished by a combination of both word-specific and general orthographic knowledge then results would take the following form: (a) for words read correctly in isolation orthographic choice performance should approach 100% (presumably due to word-specific orthographic knowledge) and (b) for words read incorrectly in isolation OC performance should be significantly above chance.

Second, we were interested in identifying other person- (e.g., phonemic awareness, word recognition, & spelling recognition) and item-factors (e.g., decodability, trigram frequency, orthographic neighborhood size, & word frequency) that influence OC performance while controlling for item-specific word recognition. We reason that if the OC task measures the quality of an orthographic representation of a specific word then person-level general measures of phonemic awareness, reading, and spelling should drop out as significant predictors of orthographic choice performance after controlling for item-specific reading skill. If however, it relies at least partly on general orthographic knowledge, then general measures of phonemic awareness, reading, and spelling ability should significantly predict the probability of making a correct orthographic choice after controlling for item-specific word recognition. On the item side, if general orthographic knowledge is involved in completing the OC task then we would expect word-level measures such as trigram frequency3, orthographic neighborhood size, and word frequency to predict individual differences in OC task performance. However, given the construction of the task, we expect decodability of the item not to predict individual differences in performance.

Third, we explored if an interaction between person- and item-characteristics would also predict variance in item performance on the OC task. Specifically, we tested the interaction between spelling recognition ability and word frequency. Our hypothesis was that one’s general orthographic aptitude (as measured by spelling ability) may moderate the benefit of seeing the word often in text (frequency) on one’s ability to acquire the word’s orthographic form.

Method

Participants and Procedure

A subset of participants from the Colorado Learning Disabilities Research Center (CLDRC) twin sample was included in the present study. The CLDRC sample is comprised of two types of twin pairs, one with and one without a school history of reading difficulty in at least one member of the pair. Twins in the sample resided within 100 miles of Denver, Colorado and were enrolled in 27 different school districts. In line with our research question, we included only the 296 participants who had complete data on tests of orthographic choice, phonemic awareness, spelling, reading, and item-specific reading of the orthographic choice target words. In our study, 31.7% of the sample was identified as a twin having a “school history of RD”, participants’ ages ranged from 8 to nearly 19 years, and gender was represented approximately equally.

Measures

Person-by-Item Measure (Dependent Measure)

Item-level orthographic knowledge.

Item-based orthographic knowledge was assessed by a forced orthographic choice task (Olson et al., 1994; Olson et al., 1989). A target word and a corresponding pseudohomophone of that word (the foil), for example, take and taik, were presented in random order on a computer screen. For some items, the target words were not decodable using common letter-sound correspondences (e.g., Rastle & Coltheart, 1999), but their pseudohomophones were always decodable (e.g., thumb-thum, muscle-mussle). For each item, participants were asked to choose the correct orthographic pattern as quickly as possible. The 80 target words were widely varied in frequency and orthographic regularity. Cronbach’s alpha for this task was .90.

Item-Specific Predictor

Item-specific word recognition.

Participants were presented the orthographic knowledge target words in a list and asked to read it aloud. Responses were scored as 1 or 0. Cronbach’s alpha for this task was .98.

Person-Level Measures

Phonemic awareness.

Phonemic awareness was measured with two phoneme deletion tasks. The first task was based on one described by Bruce (1964) in which participants hear a nonword and are asked to delete a given phoneme such that a real word is formed (i.e., prot ➔ pot). After 6 practice items, participants are asked to complete 40 test trials. There is a time limit of 4 seconds to provide the real word. The second task was similar in procedure but was based on Rosner and Simon’s (1971) auditory analysis task. This task is comprised of 5 practice items and 28 test trials.

Spelling recognition.

The Spelling subtest of the Peabody Individual Achievement Test (PIAT; Dunn & Markwardt, 1970) was used to assess spelling skill. For this subtest, participants are presented with the oral form of a word and then asked to choose between four orthographically and phonologically similar spellings. All 84 items on the test were administered unless the participant reached a ceiling of 5 incorrect out of 7 consecutive items.

Word recognition.

The word recognition variable represented the average of two measures of word reading, the Word Recognition subtest of the PIAT (Dunn & Markwardt, 1970) and an experimental Time-Limited Word Recognition Test (Olson et al., 1989; Olson et al., 1994). On the PIAT, participants attempted to read aloud 66 words in order of increasing difficulty. A ceiling rule of 5 incorrect out of 7 consecutive items was enforced. The experimental measure was a computer-administered timed test of single word reading. Responses were counted as correct if the pronunciation was accurate and initiated within 2 s of presentation. Screener items were given so that participants began the test on the appropriate item. A ceiling rule of 10 incorrect out of 20 consecutive items was enforced, unless the end of the 182-item test was reached first.

Item-Level Measures

Decodability.

The decodability variable was coded 1 if the target word was decodable according to feedforward grapheme-phoneme correspondence rules, and 0 otherwise. For example, take was coded 1 and word was coded 0. Coding was based on the grapheme-phoneme correspondences listed by Rastle and Coltheart (1999). All pseudohomophone foils were decodable, so this variable represents only target-word decodability.

Difference in trigram frequency.

The difference in mean trigram frequency was calculated in four steps. First, we used data from Solso, Barbuto, and Juel (1979) to record the frequency of each trigram in each word. Second, we summed the frequencies within each word and divided by the number of trigrams in the word (to account for different lengths of items). Third, we took the log (base 10) of the averaged values (to normalize the frequency distributions). Fourth and finally, we subtracted the log frequency of the foil from the log frequency of the target.

Orthographic neighborhood size.

Orthographic neighborhood size was obtained from the English Lexicon Project Database (Balota et al., 2007). Orthographic neighborhood size is the number of words that can be obtained by changing one letter while preserving the identity and positions of the other letters (i.e., Coltheart’s N; Coltheart, Davelaar, Jonasson, & Besner, 1977). For example, the orthographic neighbors of the word take are bake, cake, fake, hake, Jake, lake, make, rake, sake, tale, tame, tape, tare, tyke, and wake.

Word frequency.

Word frequency was obtained from the English Lexicon Project Database (Balota et al., 2007). The specific measure we used was the log-transformed frequency from the Hyperspace Analogue to Language (Lund & Burgess, 1996). The norming corpus of the HAL consists of 131 million words.

Data Analysis

Because of the wide range of ages in the current sample, all measures were adjusted for age by regressing out the effects of age and age2. In addition, raw scores were standardized to the entire CLDRC sample of twin pairs who had no school history of reading problems. The standardized and age-adjusted scores were then used for further analysis. To answer our research questions, we employed a crossed random effects item-response model, with a random effect for both persons and items. The effects were crossed rather than nested because each person attempted each item. The outcome was dichotomously scored and therefore a Bernoulli distribution was utilized. All models were fit using the lmer function contained in the lme4 in R package (Bates, Maechler, & Bolker, 2011). To produce an easily interpretable intercept, continuous variables were grand-mean centered before entry into the models.

Crossed random effects models allow us to ask questions about variability in responses across both persons and items. Therefore, person and items covariates, and interactions between them, were assessed within the same model while accounting for inter-person and inter-item variance. In this study, our aim was to examine the relation between word recognition and orthographic knowledge at the item level so that we have an estimate of the effect of correctly reading a word in isolation on the probability of choosing that word’s orthographic form. Further, we wanted to know the person- and item-characteristics that influence correct orthographic choice above and beyond the ability to read the word correctly. Specifically, after controlling for item-specific word recognition, we wanted to know whether the person characteristics of phonemic awareness (γ002), general word reading skill (γ003), and spelling skill (γ004) significantly affect orthographic choice. To further control for age effects, age (γ001) was also entered as a person covariate. We thought that the effect of spelling skill might vary randomly over words, so we included a random residual term to allow for that possibility (r021i). And after controlling for item-specific word recognition, we also wanted to know whether the item characteristics of decodability (γ005), difference in trigram frequency between the target word and the foil (γ006), frequency (γ007), and orthographic neighborhood size (γ008) affect orthographic choice. We thought that the effects of frequency and orthographic neighborhood size might vary over persons, so we included random residual terms to allow for that possibility (r011j and r012j, respectively).

Finally, after controlling for item-specific word recognition as well as all main effects of person- and item-characteristics, we wanted to know whether there is an interaction between spelling ability and word frequency (γ009). We hypothesize that individuals with superior spelling ability may have better general orthographic knowledge and therefore are more likely to choose the correct orthographic form of high-frequency words than low-frequency words based on their ability to build up orthographic representations from repeated exposures to high-frequency words. For individuals with poorer spelling skill, though, we hypothesized that there may be no difference in correct choice between high- and low-frequency words due to their poorer ability to build orthographic representations even from multiple exposures of high-frequency words.

Results

Descriptive statistics for the item, person, and item-by-person variables are in Table 1. For the items, the table reveals that 55% of the target words were decodable and the average orthographic neighborhood size was 4.06. The log values for frequency and difference in trigram frequency are not readily interpretable, but the descriptive statistics are provided in the table nonetheless. As for the sample of persons, the table shows that the average age was 11.49 years. The remaining person variables represent mean values of age-adjusted, standardized scores. Although the entire CLDRC no school history sample had means of 0 and standard deviations of 1 for these variables, our subsample statistics deviate slightly from a standardized scale because some of the twins had a school history for RD (see Olson, 2006). Across all items and all persons, the average proportion of correct responses was .83 (SD = .38) for orthographic choice and .86 (SD = .34) for item-specific word recognition. Table 2 is the correlation matrix for the person and word variables. Phonological awareness, spelling, and general word recognition were highly correlated as were correlations between whether a target word was decodable and neighborhood size and word frequency and neighborhood size.

Table 1.

Descriptive Statistics of Person and Item Variables

Variable Mean SD Min Max
Item (I = 80)
DEC .55 .50 0 1
DTF 0.20 0.62 −1.10 3.51
FREQ 9.62 2.08 0.00 13.27
NSIZE 4.06 5.29 0 22
Person (J = 296)
Age 11.49 2.64 8.06 18.98
PA −0.61 1.41 −5.29 1.84
SPELL −0.58 1.17 −3.89 2.66
READ −0.73 1.43 −4.96 2.33
Item-by-Person (r = 23,679)
Orthographic Choice .83 .38 0 1
WORD_REC .86 .34 0 1
Open in a new tab

Note. DEC = decodability; DTF = difference in (log) trigram frequency; FREQ = (log) frequency; NSIZE = orthographic neighborhood size; PA = phonological awareness; READ = (general) reading. SPELL = spelling; WORD_REC = item-specific word recognition.

Table 2.

Correlation Table for Person and Word Variables

Person (J = 296)
Variable 1 2 3 4
1. Age —
2. PA .09 —
3. Spelling .05 .72* —
4. Word reading .02 .72* .88* —
Word (I=80)
Variable 1 2 3 4
1. DEC —
2. DTF −.10 —
3. NSIZE .40* .07 —
4. FREQ .03 .01 .24* —
Open in a new tab

PA = Phonemic awareness; DEC = decodability; DTF = difference in (log) trigram frequency; FREQ = (log) frequency; NSIZE = orthographic neighborhood size.

*

p < .05

Model estimation began with an unconditional model to partition the variance in responses and to report the average response across all items and all persons in our sample (Model 1). Results are listed in Table 3. Model 1 indicated that the probability of a correct orthographic choice for the average person and the average item was .89 (pji = 11+exp(−γ^000) = 11+exp(−2.06)). Variability around that average was evident for both persons and items. There are two ways of describing variability among units, the intraclass correlation (ICC) and the 95% plausible value range (Raudenbush & Bryk, 2002). The ICC between persons (conditional on item difficulty) was .17 and between items (conditional on person ability) was .18 (Cho & Rabe-Hesketh, 2011). The 95% plausibility value range for persons was .55-,98 and .54-,98 for items (Raudenbush & Bryk, 2002). Because of this variability, we proceed with our research questions aimed at explaining the variability among persons and words.

Table 3.

Results for All Crossed Random-Effects Models (J = 296, I = 80, r = 23,679)

Model
1. Unconditional Model 2. Item-Specific Recognition Model 3. Person & Item Main Effects Model 4. Interaction Model
Est. SE z Est. SE z Est. SE z Est. SE z
Fixed Effects
Intercept (γ000) 2.06 0.12 16.55 1.33 0.10 13.85 1.73 0.11 15.16 1.74 0.11 15.23
WORD_REC (γ100) 0.81 0.09 9.35* 0.34 0.08 4.37* 0.34 0.08 4.35*
Age (γ001) 0.20 0.02 12.84* 0.19 0.02 12.82*
PA (γ002) −0.10 0.04 −2.61* −0.11 0.04 −2.82*
READ (γ003) 0.10 0.06 1.69 0.10 0.06 1.79
SPELL (γ004) 0.36 0.07 5.01* 0.38 0.07 5.29*
DEC (γ005) 0.19 0.15 1.29 0.18 0.15 1.23
DTF (γ006) 0.15 0.12 1.19 0.15 0.12 1.21
FREQ (γ007) 0.14 0.12 3.81* 0.15 0.04 4.18*
NSIZE (γ008) 0.09 0.02 5.59* 0.09 0.02 5.23*
SPELL X FREQ (γ009) 0.02 0.01 2.23*
Random Effects
Item Intercept (σr0202) 0.95 0.36 0.21 0.21
WORD_REC (σr1202) 0.27 0.13 0.13
SPELL (σr0212) 0.02 0.01
Person Intercept (σr0102) 0.90 0.52 0.34 0.34
WORD_REC (σr1102) 0.17 0.05 0.05
FREQ (σr0112) 0.00 0.00
NSIZE (σr0122) 0.00 0.00
Open in a new tab

Note. Each model was compared to the one immediately preceding it. WORD_REC = item-specific word recognition; PA = phonological awareness; READ = (general) reading; SPELL = spelling; DEC = decodability; DTF = difference in (log) trigram frequency; FREQ = (log) frequency; NSIZE = orthographic neighborhood size.

*

p < .05

In Model 2 we added the item-specific word recognition variable (including its fixed term, a person variance and covariance term, and an item variance and covariance term) to Model 1. Item-specific word reading accuracy was a significant predictor of correctly selecting the correct response on orthographic choice (ŷ100 = 0.81, z = 9.35). As stated, if the OC task were merely a word reading task, the probability of choosing the correct orthographic form of a word would be near .50 (chance) when the word was read incorrectly but near 1.00 (perfect) when the word was read correctly. Model 2 allows us to estimate these probabilities. The probability of choosing the correct orthographic form when the word was pronounced incorrectly was .79; whereas it was .90 when the word was pronounced correctly. Results imply that the OC task is not simply a word recognition task and tend to suggest that general orthographic knowledge may influence the choice task when there is not a complete word-specific representation. The probabilities also give reason to explore the other factors that are associated with persons and items that may contribute to performance on the OC task.

Model 3 was estimated to evaluate the effects of various person- and item-characteristics on orthographic choice after controlling for item-specific word recognition. To address the question of whether the OC task is merely a word recognition task, we were especially interested in the effect of general word recognition ability in this model. Age was entered into the model is a covariate along with phonemic awareness, reading skill, and spelling recognition skill on the person side and decodability, difference in trigram frequency, frequency, and orthographic neighborhood size on the word side. Results indicate that controlling for item-specific word recognition and various person and item features, spelling recognition skill was significantly related to orthographic choice. The significant general spelling effect along with the results of Model 2 provides evidence against the argument that the OC task only measures word recognition skill. General reading skill was not a significant predictor of orthographic choice in this model. Model 3 also indicated that controlling for the other covariates in the model, the lower a person’s phonemic awareness, the higher the (log-odds of the) probability of choosing a word’s correct orthographic form. Our explanation for the negative coefficient associated with phonemic awareness is that all of the foils were fully decodable whereas that was true for only 55% of the target words. Results suggest that when an item is unfamiliar, a person with high phonemic awareness may have relied on phonological recoding of the target and foil items, and ultimately chose the foil more often because of its decodability.

After controlling for item-specific word recognition and person- and item-covariates, higher word frequencies and larger orthographic neighborhoods were associated with higher (log-odds of) probabilities of making a correct orthographic choice. The binary indicator of decodability of the target word was not a significant predictor, nor was the difference in trigram frequency between the target word and the foil. Prior research has found frequency and orthographic neighborhood size to be important predictors of orthographic knowledge (Lété, Peereman, & Fayol, 2008; Martinet, Valdois, & Fayol, 2004) and the present study extends the literature by concluding that these relations hold even after accounting for whether a person is able to read the word.

In Model 4, we tested the interactions between spelling skill and word frequency. We wanted to explore whether there was a larger effect of word frequency for those with superior spelling skill than those with lower spelling skill. The spelling by word frequency interaction was statistically significant. To aid our interpretation of the interaction, we used the online calculator provided by Preacher, Curran, and Bauer (2006). We found that the region of significance for the frequency variable was for values of age-regressed standardized spelling scores below -2.81; representing the absolute lower end of the distribution (i.e., only one person in our sample). In other words, there is a significant effect of word frequency on correct orthographic choice except for people at the extreme lower end of the spelling ability distribution. This follow-up result leads us to conclude that there is a meaningful main effect of word frequency that exists across the continuum of spelling ability.

Finally, we replicated the models with only items that were not decodeable (I = 36). Given that a majority of the 80 items on the OC task are decodeable (I = 44) it is quite possible that item-specific word recognition could be correctly completed by simply decoding the word as opposed to relying on item-specific orthographic knowledge. To help rule out that the imperfect correlation between OC task and item-specific word recognition was simply due to phonological decoding we removed the decodable items and reran the models (i.e., without the decodeable variable). Results of the models are presented in Table 4. Overall the two models are quite similar with item-specific word recognition, age, phonemic awareness, spelling recognition, word frequency, and orthographic neighborhood size being significant predictors of orthographic choice item performance. The two models differed with regard to general word recognition as a predictor with it only being a significant predictor in the reduced model with only non-decodable words. For the reduced model the probability of a correct orthographic choice for the average person and the average item was .84, for a correct item-specific word recognition response it was .86, and for an incorrect item-specific word recognition response it was .74. Results of the reduced model seem to corroborate the full model indicating that OC task is not simply a word recognition task and further suggest that general orthographic knowledge may influence the choice task when there is not a complete word-specific representation.

Table 4.

Results for All Crossed Random-Effects Models for Non-decodable Words Only (J = 296, I = 36, r = 10,655)

Model
1. Unconditional Model 2. Item-Specific Recognition Model 3. Person & Item Main Effects Model 4. Interaction Model
Est. SE z Est. SE z Est. SE z Est. SE z
Fixed Effects
Intercept (γ000) 1.68 0.11 12.2 1.03 0.13 7.38 1.71 0.17 10.18 1.70 0.17 9.88
WORD_REC (γ100) 0.80 0.14 5.60* 0.32 0.16 2.84* 0.31 0.12 2.75*
Age (γ001) 0.21 0.02 12.84* 0.21 0.02 12.81*
PA (γ002) −0.11 0.04 −2.71* −0.11 0.04 −2.74*
READ (γ003) 0.12 0.06 2.18* 0.13 0.06 2.16*
SPELL (γ004) 0.31 0.08 4.14* 0.29 0.08 3.53*
DTF (γ006) 0.11 0.19 0.59 0.11 0.19 0.59
FREQ (γ007) 0.18 0.07 2.45* 0.22 0.08 2.89*
NSIZE (γ008) 0.10 0.04 2.47* 0.09 0.04 2.09*
SPELL X FREQ (γ009) 0.04 0.02 2.14*
Random Effects
Item Intercept (σr0202) 0.96 0.35 0.27 0.28
WORD_REC (σr1202) 0.36 0.18 0.18
SPELL (σr0212) 0.01 0.01
Person Intercept (σr0102) 0.77 0.49 0.28 0.29
WORD_REC (σr1102) 0.06 0.01 0.01
FREQ (σr0112) 0.01 0.01
NSIZE (σr0122) 0.00 0.00
Open in a new tab

Note. Each model was compared to the one immediately preceding it. WORD_REC = item-specific word recognition; PA = phonological awareness; READ = (general) reading; SPELL = spelling; DEC = decodability; DTF = difference in (log) trigram frequency; FREQ = (log) frequency; NSIZE = orthographic neighborhood size.

*

p < .05

Discussion

As we described at the outset, researchers have suggested the OC task measures word recognition rather than a distinct orthographic process, as readers who make correct orthographic choices probably have highly-specified orthographic representations for those words, formed through orthographic learning (Nation & Castles, 2017). Our first research question, therefore, asked whether item-specific word recognition accuracy would explain a majority of the item-level variance in OC task performance. Results indicated that when an individual named the target words correctly, they had a 90% chance of making the right choice, suggesting that item-specific word recognition and performance on the OC task are one in the same. What was somewhat surprising was the fact that when individuals named target words incorrectly, they still had a 79% chance of making a correct orthographic choice for that word (i.e., well above chance). We interpret these results as supporting multiple forms of orthographic knowledge (i.e., word-specific and general), conditionalized on word-specific reading performance, that influence the OC task. Specifically, when a reader has a complete representation of a word (i.e., word read correctly in isolation) then the OC task appears to assess word-specific orthographic knowledge. However, when a reader has an incomplete orthographic representation of the word (inability to read the word in isolation) we hypothesize that the OC task allows general orthographic knowledge to influence task performance through the use of incomplete or fuzzy orthographic knowledge about the word, knowledge about sublexical orthographic features of the language, and/or other unspecified orthographic knowledge. Which type(s) of general orthographic knowledge readers rely on when a word-specific orthographic representation is not available and whether this varies across readers is impossible to determine in the current study. Results are in keeping with arguments by Conrad, Harris, and Williams (2013) that orthographic knowledge is multi-dimensional, consisting of both word specific and general orthographic knowledge.

Second, we were interested in the person- and item-characteristics that predict OC task performance beyond item-specific word reading. In terms of person characteristics, we found no relation between general word recognition and orthographic choice, but we did observe significant associations between spelling recognition skill and phonological awareness skill (coefficient being negative) and orthographic choice. That general spelling recognition was a significant predictor of OC is not altogether surprising given that both tasks require the ability to decide what letter strings look acceptable and which do not. We were initially surprised by the significant and negative effect of phonemic awareness on OC task performance; however in looking carefully at the items we recognized that all of the foils were fully decodable whereas only 55% of the target words were decodable. Results suggest that when an item is unfamiliar, a person with high phonemic awareness may have relied on phonological recoding of the target and foil items, and ultimately chose the foil more often because of its full decodability. In terms of item characteristics, we found individuals were more likely to make correct orthographic choices for higher frequency words and those with larger orthographic neighborhoods. While the word frequency effect was expected, the significant effect of orthographic neighborhood size was less anticipated and more interesting. This result suggests that orthographic knowledge may develop probabilistically, reflected in the fact that words with larger orthographic neighborhoods were selected more accurately and perhaps shared sublexical properties that allow general orthographic knowledge to influence OC task performance. Importantly, trigram frequency differences between the target word and the foil was not a significant predictor. Suggesting that shared orthography as measured by orthographic neighborhood size may be more relevant than general trigram frequency when making selections on the OC task.

Unresolved—and Important—Issues

This study does not resolve whether orthographic knowledge skill is a process etiologically distinct from phonological processing, phonological recoding, and reading experience (see Burt, 2006). We make no claim that the task can show whether readers vary in their ability to form, store, and access orthographic representations. It only shows that differences in readers’ general orthographic knowledge may exist. The OC task cannot show how these differences are obtained or what they represent, as Deacon et al. (2012) noted. It is possible that much of the difference in the quality of readers’ general orthographic representations can be explained by readers’ general ability to form, store, and access information (i.e., general cognitive ability) and their reading experience. In this study, we did not have access to measures of this kind. It is possible that differences in general orthographic knowledge are explained by readers’ opportunities to build this type of knowledge and their ability to profit from these opportunities. We agree with Deacon et al. (2012), who suggested that new measures of orthographic skill should be developed to examine orthographic knowledge itself, not just the outcomes of it.

We recognize that in evaluating our first research question (i.e., item-specific word recognition accuracy was necessary and sufficient for making a correct orthographic choice) we must assume our measures are perfectly reliable. With perfect reliability we can logically reason that if item-specific word recognition is necessary, the probability of correctly choosing the orthographic form of a word when a student cannot read a word should be near chance; whereas the probability of correctly choosing the orthographic form of a word when a student can read the word should approach 1.00. However, the reliability of our measures was not perfect (i.e., Cronbach’s alpha was .90 for the OC task and .98 for the item-specific word reading task) and therefore we must grapple with the role of measurement error in interpreting the first research question. Given that the probability of correctly selecting the correct item on the OC task was .90 when the word was pronounced correctly, it is quite possible that with perfect measures this probability could approach 1.0. However, applying a similar argument, it is quite possible that with a lack of measurement error the probability of a correct item response on the OC task could increase above the .79 reported here. Thus, we are mindful of the potential bias created by measurement error when we suggest that out results support multiple forms of orthographic knowledge (i.e., word-specific and general) that may influence the OC task.

Conclusion

Overall, our data suggest that performance on the OC task may partly rely on general orthographic knowledge, but only for certain items where an item-specific representation is not available. Contrary to the idea that the OC task relies entirely on word-specific orthographic knowledge, a form of knowledge strongly linked to word recognition (Burt, 2006; Vellutino et al., 1994, 1995), we found evidence that—after accounting for word-specific reading skill—there was still variability in individuals’ orthographic choices to explain. The covariates that explained this variability strongly suggested that readers make orthographic choices based on orthographic knowledge that is separate from phonological knowledge. And thus, we have argued here that the OC task also measures some type of general orthographic knowledge. For the purpose of measuring orthographic knowledge, the OC task appears to be an appropriate selection as long as the user understands how word-specific knowledge moderates the need to evoke more general orthographic knowledge to make the correct item choice.

Acknowledgments

This research was supported in part by Grant P50HD027802 awarded to The University of Colorado and Grant P20HD091013 awarded to Florida State University by Eunice Kennedy Shriver National Institute of Child Health and Human (NICHD). The content is solely the responsibility of the authors and does not necessarily represent the official view of NICHD.

Footnotes

Publisher's Disclaimer: This Author Accepted Manuscript is a PDF file of a an unedited peer-reviewed manuscript that has been accepted for publication but has not been copyedited or corrected. The official version of record that is published in the journal is kept up to date and so may therefore differ from this version.

1

We use the term orthographic knowledge, as opposed to orthographic processing, to emphasize the accumulation of knowledge resulting from orthographic learning (Nation & Castles, 2017).

2

We acknowledge that our item-specific word-recognition task is not a pure measure of item-specific orthographic knowledge and therefore inferences that any item-level variance in the orthographic choice task not accounted for by this task is attributable to general orthographic knowledge must be conditionalized on this possible mismatch between measure and construct.

3

We included trigram frequency as a word-level predictor based on work by Siegel, Share, & Geva (1995) indicating children base orthographic decisions in part on the probable sequences and positions of letters that appear within the English corpus. In this study trigram frequency served as a proxy for sensitivity to probable letter positions, a measure of more general orthographic knowledge.

References

  1. Adams MJ (1994). Beginning to read: Thinking and learning about print. Cambridge, MA: MIT Press. [Google Scholar]
  2. Balota DA, Yap MJ, Cortese MJ, Hutchison KA, Kessler B, Loftis B, … Treiman R (2007). The English Lexicon Project. Behavior Research Methods, 39, 445–459. [DOI] [PubMed] [Google Scholar]
  3. Barker TA, Torgesen JK, & Wagner RK (1992). The role of orthographic processing skills on five different reading tasks. Reading Research Quarterly, 27, 334–345. [Google Scholar]
  4. Bates D, Maechler M, & Bolker B (2011). Ime4: Linear mixed-effects models using S4 classes. R package version 0.999375–39. http://CRAN.R-project.org/package=lme4 [Google Scholar]
  5. Blachman B (2000). Phonological awareness In: Kamil ML, Mosenthal PB, Pearson PD, & Barr R (Eds.), Handbook of reading research (Vol. 3, pp. 483–502). [Google Scholar]
  6. Bruce DJ (1964). The analysis of word sounds. British Journal of Educational Psychology, 34, 158–170. [Google Scholar]
  7. Burt JS (2006). What is orthographic processing skill and how does it relate to word identification in reading? Journal of Research in Reading, 29, 400–416. [Google Scholar]
  8. Cassar M, & Treiman R (1997). The beginnings of orthographic knowledge: Children’s knowledge of double letters in words. Journal of Educational Psychology, 89, 631–644. [Google Scholar]
  9. Castles A, & Nation K (2006). How does orthographic learning happen? In Andrews S (Ed.), From inkmarks to ideas: Current issues in lexical processing (pp. 151–179). Hove, England: Psychology Press. [Google Scholar]
  10. Cho S-J, & Rabe-Hesketh S (2011). Alternating imputation posterior estimation of models with crossed random effects. Computational Statistics and Data Analysis, 55, 12–25. [Google Scholar]
  11. Coltheart M, Davelaar E, Jonasson JT, & Besner D (1977). Access to the internal lexicon. Dornick S (ed.) Attention and performance, volume VI, 535–556. [Google Scholar]
  12. Conrad NJ, Harris N, & Williams J (2013). Individual differences in children’s literacy development: The contribution of orthographic knowledge. Reading and Writing, 26(8), 1223–1239. [Google Scholar]
  13. Cunningham AE, Perry KE, & Stanovich KE (2001). Converging evidence for the concept of orthographic processing. Reading and Writing: An Interdisciplinary Journal, 14, 549–568. [Google Scholar]
  14. Cunningham AE, Perry KE, Stanovich KE, & Share DL (2002). Orthographic learning during reading: Examining the role of self-teaching. Journal of experimental child psychology, 82(3), 185–199. [DOI] [PubMed] [Google Scholar]
  15. Cunningham AE, & Stanovich KE (1990). Assessing print exposure and orthographic processing skill in children: A quick measure of reading experience. Journal of Educational Psychology, 82, 733–740. [Google Scholar]
  16. Cunningham AE, Stanovich KE, & West RF (1994). Literacy environment and the development of children’s cognitive skills In: Assink EΜH (Ed.), Literacy environment and the development of children’s cognitive skills (pp. 70–90). [Google Scholar]
  17. Deacon SH (2012). . Sounds, letters and meanings: The independent influences of morphological and orthographic skills on early word reading accuracy. Journal of Research in Reading, 55(4), 456–475. [Google Scholar]
  18. Deacon SH, Benere J, & Castles A (2012). The chicken or egg? Untangling the relationship between orthographic processing skill and reading accuracy. Cognition, 122, 110–117. doi: 10.1016/j.cognition.2011.09.003 [DOI] [PubMed] [Google Scholar]
  19. Dunn LM, & Markwardt FC (1970). Examiner’s manual: Peabody Individual Achievement Test. Circle Pines, MN: American Guidance Service. [Google Scholar]
  20. Ehri LC (2005). Learning to read words: Theory, findings, and issues. Scientific Studies of Reading, 9, 167–188. [Google Scholar]
  21. Ehri LC (2014). Orthographic mapping in the acquisition of sight word reading, spelling memory, and vocabulary learning. Scientific Studies of Reading, 75(1), 5–21. [Google Scholar]
  22. Elleman AM, Steacy LM, & Compton DL (2019). The role of statistical learning in word reading and spelling development: More questions than answers. Scientific Studies of Reading, 23, 1–7. [DOI] [PMC free article] [PubMed] [Google Scholar]
  23. Frith U (1980). Cognitive processes in spelling. London: Academic Press. [Google Scholar]
  24. Gilbert JK, Compton DL, & Kearns DM (2011). Word and person effects on decoding accuracy: A new look at an old question. Journal of Educational Psychology, 103, 489–507. [DOI] [PMC free article] [PubMed] [Google Scholar]
  25. Hagiliassis N, Pratt C, & Johnston M (2006). Orthographic and phonological processes in reading. Reading and Writing, 19, 235–263. [Google Scholar]
  26. Kim Y-S, Petscher Y, Foorman BR, & Zhou C (2010). The contributions of phonological awareness and letter-name knowledge to letter-sound acquisition – a cross-classified multilevel approach. Journal of Educational Psychology, 102, 313–326. [Google Scholar]
  27. Lété B, Peereman R, & Fayol M (2008). Consistency and word-frequency effects on spelling among first- to fifth-grade French children: A regression-based study. Journal of Memory and Language, 58, 952–977. doi: 10.1016/j.jml.2008.01.001 [DOI] [Google Scholar]
  28. Lund K, & Burgess C (1996) Producing high-dimensional semantic spaces from lexical co-occurrence. Behavior Research Methods, Instruments, & Computers, 28, 203–208. [Google Scholar]
  29. Manis FR, Doi LM, & Bhadha B (2000). Naming speed, phonological awareness, and orthographic knowledge in second graders. Journal of Learning Disabilities, 33, 325–333,374. doi: 10.1177/002221940003300405 [DOI] [PubMed] [Google Scholar]
  30. Manis FR, Custodio R, & Szeszulski PA (1993). Development of phonological and orthographic skill: A 2-year longitudinal study of dyslexic children. Journal of Experimental Child Psychology, 56, 64–86. [DOI] [PubMed] [Google Scholar]
  31. Martinet C, Valdois S, & Fayol M (2004). Lexical orthographic knowledge develops from the beginning of literacy acquisition. Cognition, 91, B11–B22. [DOI] [PubMed] [Google Scholar]
  32. Nation K, & Castles A (2017). Putting the learning in to orthographic learning In Cain K, Compton D, & Parrila R (Eds.), Theories of reading development (pp. 147–168). Amsterdam, The Netherlands: John Benjamins. [Google Scholar]
  33. Nation K, Angells P, & Castles A (2007). Orthographic learning via self-teaching in children learning to read English: Effects of exposure, durability, and context. Journal of Experimental Child Psychology, 96( 1), 71–84. [DOI] [PubMed] [Google Scholar]
  34. National Institute of Child Health and Human Development (NICHD). (2000). Report of the National Reading Panel Teaching children to read: An evidence-based assessment of the scientific research literature on reading and its implications for reading instruction: Reports of the subgroups (NIH Publication No. 00-4754). Washington, DC: U.S. Government Printing Office; Retrieved November 11, 2005, from http://www.nichd.nih.gov/publications/nrp/report.htm [Google Scholar]
  35. Olson RK (2006). Genes, environment, and dyslexia the 2005 Norman Geschwind memorial lecture. Annals of Dyslexia, 56(2), 205–238. [DOI] [PubMed] [Google Scholar]
  36. Olson RK, Forsberg H, & Wise B (1994). Genes, environment, and the development of orthographic skills In Berninger VW (Ed.), The varieties of orthographic knowledge I: Theoretical and developmental Issues (pp. 27–71). Dordrecht, the Netherlands: Kluwer. [Google Scholar]
  37. Olson R, Wise B, Conners F, Rack J, & Fulker D (1989). Specific deficits in component reading and language skills: Genetic and environmental influences. Journal of Learning Disabilities, 22, 339–348. [DOI] [PubMed] [Google Scholar]
  38. Piasta SB, & Wagner RK (2010). Learning letter names and sounds: Effects of instruction, letter type, and phonological processing skill. Journal of Experimental Child Psychology, 105, 324–344. [DOI] [PMC free article] [PubMed] [Google Scholar]
  39. Preacher KJ, Curran PJ, & Bauer DJ (2006). Computational tools for probing interaction effects in multiple linear regression, multilevel modeling, and latent curve analysis. Journal of Educational and Behavioral Statistics, 31, 437–448. [Google Scholar]
  40. Rastle K, & Coltheart M (1999). Serial and strategic effects in reading aloud. Journal of Experimental Psychology: Human Perception and Performance, 25, 482–503. [DOI] [PubMed] [Google Scholar]
  41. Raudenbush SW, & Bryk AS (2002). Hierarchical linear models: Applications and Data Analysis Methods (2nd ed.). Thousand Oaks, CA: Sage. [Google Scholar]
  42. Rosner J, & Simon D (1971). The auditory analysis test: An initial report. Journal of Learning Disabilities, 4, 384–392. [Google Scholar]
  43. Share DL (2004). Orthographic learning at a glance: On the time course and developmental onset of self-teaching. Journal of Experimental Child Psychology, 87, 267–298. [DOI] [PubMed] [Google Scholar]
  44. Siegel LS, Share D, & Geva E (1995). Evidence for superior orthographic skills in dyslexics. Psychological science, 6(4), 250–254. [Google Scholar]
  45. Solso RL, Barbuto PF Jr., & Juel CL (1979). Bigram and trigram frequencies and versatilities in the English language. Behavior Research Methods and Instrumentation, 11, 475–484. [Google Scholar]
  46. Stanovich KE (1986). Matthew effects in reading: Some consequences of individual differences in the acquisition of literacy. Reading Research Quarterly, 21, 360–407. [Google Scholar]
  47. Stanovich KE, & West RF (1989). Exposure to print and orthographic processing. Reading Research Quarterly, 402–433. [Google Scholar]
  48. Stanovich KE, West RF, & Cunningham AE (1991). Beyond phonological processes: Print exposure and orthographic processing In: Brady SA, & Shankweiler DP (Eds.), Phonological processes in literacy: A tribute to Isabelle Y Liberman, (pp. 219–235). [Google Scholar]
  49. Van den Noortgate WV, De Boeck P & Meulders M (2003). Cross-classification multilevel logistic models in psychometrics. Journal of Educational and Behavioral Statistics, 28, 369–386. http://www.jstor.org/stable/3701341 [Google Scholar]
  50. Vellutino FR, Scanlon DM, & Chen RS (1995). The increasingly inextricable relationship between orthographic and phonological coding in learning to read: Some reservations about current methods of operationalizing orthographic coding In Berninger VW (Ed.), The varieties of orthographic knowledge, 2: Relationships to phonology, reading, and writing, (pp. 47–111). Dordrecht, the Netherlands: Kluwer Academic Publishers. [Google Scholar]
  51. Vellutino FR, Scanlon DM, & Tanzman MS (1994). Components of reading ability: Issues and problems in operationalizing word identification phonological coding, and orthographic coding (pp. 279–332). In Lion GR, Frames of reference for the assessment of learning disabilities: New views on measurement issues. Baltimore, MD: Brookes. [Google Scholar]
  52. Wagner R,K, & Barker TA. (1994). The development of orthographic processing ability In Berninger VW (Ed.), The varieties of orthographic knowledge: Theoretical and developmental issues (pp. 243–276). Springer, Dordrecht. [Google Scholar]
  53. Wang H-C, Nickels L, Nation K, & Castles A (2013). Predictors of orthographic learning of regular and irregular words. Scientific Studies of Reading, 17, 369–384. [Google Scholar]

RESOURCES