Effects of Semantic Reinforcement, Semantic Discrimination and Affix Frequency on New Word Learning in Skilled and Less Skilled Readers in Grades Six to Twelve.

Abstract

Two approaches to word learning were investigated in 1,215 6^th to 12^th grade students. Definitions were provided followed by either two sentences that were semantically correct exemplars, called semantic reinforcement learning or by one correct sentence and a contrasting incorrect sentence (i.e., example followed by a structurally-aligned non-example), called semantic discrimination learning. Type of learning was blocked and examples and non-examples were explained. Effects of affix frequency were also assessed. Students were taught words followed by assessments of abilities to recall the meaning of the word immediately after learning it, to choose the correct word from amongst distractors to match a given definition after all words had been instructed, and, to judge the semantic veracity of new sentences containing taught words one to three days later. Explanatory item response models were used to predict word learning using student and item characteristics along with their interactions. Few grade-related differences emerged. Higher frequency affixes were generally beneficial for learning and retention across comprehension skill levels and measures. Immediate recall of word meanings was facilitated by semantic reinforcement learning. In contrast, performance after all of the words had been instructed was facilitated by semantic discrimination learning, but only for more highly skilled comprehenders. The ability to learn the meanings of new words accounted for unique variance on one measure of reading comprehension controlling for decoding, previously acquired vocabulary knowledge, and working memory. Results are discussed with reference to models of vocabulary learning and implications for vocabulary instruction for adolescents.

Vocabulary knowledge is important for reading comprehension. In the preschool and early elementary grades it predicts reading comprehension several years later, even after controlling for reading comprehension from earlier time-points (de Jong & van der Leij, 2002; Oakhill & Cain, 2012; Quinn et al., 2015; Storch & Whitehurst, 2002; Verhoeven & van Leeuwe, 2008). Vocabulary knowledge also has considerable indirect effects on reading comprehension through inference-making (Ahmed et al., 2016; Cromley & Azevedo, 2007; Currie & Cain, 2015), literal text comprehension (Silva & Cain, 2015), and strategy use (Cromley & Azevedo, 2007). In the secondary grades when informational text is increasingly used for acquiring academic content, vocabulary knowledge is one of the strongest contributors to reading comprehension (Ahmed et al., 2016; Cromley & Azevedo, 2007). Despite the fact that informational text contains many specialized and morphologically complex words (Nagy & Scott, 2000), which are known to create a source of significant text difficulty for readers in the secondary grades (McNamara et al., 2012), relatively little is known about what affects new word learning in these older students.

Studies of individual differences show that less skilled readers have a range of semantic deficits. Less skilled comprehenders with adequate word decoding have difficulties rapidly accessing a broad range of semantic connections about and between words (Nation & Snowling, 1999). Compared to their more skilled peers, adults with poorer reading comprehension show a smaller neural effect (in evoked response potentials) of semantic relatedness between words (e.g. Landi & Perfetti, 2007; Perfetti & Stafura, 2014). Less skilled comprehenders have difficulties acquiring new word meanings implicitly from context (Cain et al., 2004b), as well as explicitly through instruction. For example, 8–9 year-old skilled comprehenders learned to associate new words with pictures of novel objects and were better able to retain semantic information related to these new words compared to their peers with poorer comprehension (Nation et al., 2007).

There may be several sources of variability in vocabulary learning including the conditions under which new words are learned, the properties of the words themselves, and cognitive abilities of the learner. In the current study, we investigated several of these potential sources of variability in new word learning in adolescents. We tested the effects of the learning context (i.e. type of instruction), morphological features of words (i.e., affix frequency), and the contributions of two child characteristics - reading comprehension level and working memory. The literature on these potential sources of individual differences in new word learning is reviewed below.

Instructional Approaches to New Word Learning

The conditions under which words are instructed contributes to new word learning in various ways. Although many new words are learned incidentally from context, meta-analyses show that explicit vocabulary instruction leads to increases in vocabulary knowledge and reading comprehension and may be particularly important for struggling readers. For example, Stahl & Fairbanks (1986) found a large effect of instruction on understanding newly taught words when those words were tested in new passages. Elleman and colleagues (2009) found moderate effects of vocabulary instruction on “custom” or more proximal measures that looked at passage-level comprehension with larger effects for less skilled versus more skilled readers. Wright & Cervetti (2017) found that even brief instruction that provided both word definitions and experiences with words in context (like the design of the current study) resulted in learning; however, longer duration instruction (e.g., Lesaux et al., 2014) at over 60 minutes per word with several different types of word activities was associated with the largest effects. When comparing active versus less active instruction conditions within studies (looking up meaning in dictionary vs. provided with meaning and completing activities with words in context), more active instruction was associated with larger effects.

One learning model that includes both a context dependent learning process and learning through definitions has been proposed by Reichle & Perfetti (2003). This instance-based model combines the assumption of instance-based word memories with resonance processes that activate these memories each time the word is encountered in a new context. According to this model of incremental word learning, word knowledge accrues with each experience with the word in context. With each encounter, memory traces of prior contexts affect the processing of a new encounter whereby overlapping memory traces resonate and become fragments of decontextualized knowledge. The more encounters with a word across a variety of contexts, the greater the opportunity for memory traces to resonate and become decontextualized and, therefore, less dependent on context. Thus, several contexts should enable more abstract learning than an equal number of experiences with a single context. According to this framework, definitions also support learning of abstract information because they provide core word features in a single learning event.

In a study to test this instance-based model (Bolger et al., 2008), undergraduate students were taught the meaning of 72 rare words that they encountered four times in four different conditions: In the varied (4-contexts) condition, the word appeared in a novel sentence each time; in the repeated (1-context) condition, the word appeared in the same sentence four times; in the definition conditions, the definition appeared either along with the varied context sentences; or along with the repeated context sentences. Exposure to variable contexts led to better accuracy on the meaning generation task than did equivalent exposure to a single context. Furthermore, the combination of definitions with sentence contexts was more effective than context alone. Finally, the ability to learn new word meanings was correlated with reading comprehension.

Another type of learning context that may also be helpful for learning new words is non-examples. Non-examples are effective for identifying word concepts when they share commonalities and alignable differences with paired examples (Markman & Gentner, 1993; Markman & Wisniewski, 1997). This method is consistent with structure-mapping theory in which two types of analogical processing (abstraction and contrast) are used for learning (Gentner et al., 2009). When two analogous, but contrasting examples are used their commonalities and differences become more apparent for the learner. When applied to word learning, structure-mapping theory suggests that when teaching new word meanings using example and non-example sentences, it is most effective to use sentence pairs that are highly similar, such that their differences are easily noticed. This is consistent with Markle’s (1975) approach for teaching word meaning by contrasting non-example sentences that differ conceptually from example sentences by changing one critical attribute.

Positive effects of comparing pairs of highly similar items with alignable differences in multiple domains of learning are well established in studies of adult learners (e.g., Gentner & Markman, 1997; Gick & Holyoak, 1983; Medin et al., 1993; Wolff & Gentner, 2000) and in young children (e.g., Waxman & Klibanoff, 2000; Loewenstein & Gentner, 2001). It has been shown to be a powerful method in word learning (Gentner & Namy, 2004), and in other cognitive domains, such as decision-making/judgment and spatial concept learning (e.g., Gentner et al., 2009; Tversky & Kahneman, 1986).

Although non-examples have been used in studies to teach new words (Beck & McKeown, 1983; Klausmeier et al., 1974; Markle, 1975) this type of discrimination learning has not been compared to the use of varied sentence examples (Bolger et al., 2008), nor have such comparisons been made among students with different levels of reading comprehension. This study compares these two types of word learning approaches in a large sample of adolescents who vary in their reading comprehension levels.

Morphology and New Word Learning

Morphological features of words such as affix frequency may provide semantic cues that can help in the learning of new words. Texts in the upper grades contain words that are morphologically complex with estimates that about 60% of these words have parts (morphemes) that carry meaning and that can be used to infer meaning (Nagy & Scott, 2000). Morphological knowledge is reliably related to reading comprehension. For example, morphological knowledge in the elementary grades is related to children’s vocabulary knowledge (Carlisle, 2007; Singson et al., 2000) and to their reading comprehension (e.g., Deacon & Kirby, 2004; Nagy et al., 2006). Controlling for several reading and reading-related skills (including vocabulary knowledge), morphological analysis, the ability to use morphology to decode the meaning of new complex words, predicted gains in reading comprehension between grades 3 and 4 (Levesque et al., 2018). Previous studies have not examined the effect of morphological frequency (i.e., the ability to use prefixes and suffixes that are more or less frequent in English) on the ability of skilled and less skilled adolescent comprehenders to learn new words in different learning conditions even though these morphologically complex words become more commonly used in secondary grade texts (Nagy & Scott, 2000).

Working Memory and New Word Learning

The relations of working memory and vocabulary knowledge (both receptive and expressive) are small to moderate in size (meta-analysis of Peng et al., 2018). Explanations for this relation range from the importance of updating working memory during new word learning (e.g., Martin et al., 2019), the need to bind orthographic, phonological and semantic representations (Ehri, 2014; Perfetti, 2007; Rosenthal & Ehri, 2008), to a non-directional relationship based on the fact that, at least for verbal working memory and vocabulary, both are language skills (MacDonald & Christiansen, 2002). One experimental study of new word learning from context showed that children with poor working memory had considerable difficulty in learning the meaning of new words when context cues were not immediately adjacent to the new word (Cain et al., 2004b), suggesting that memory processes may also be important for learning new word meanings from text. In terms of the conditions under which new words are learned, it is possible that comparative processes involved in learning through examples and non-examples may require greater working memory resources than when learning is based on consistent semantic exemplars; however, the relation of working memory to vocabulary learning under different learning conditions has not been tested.

Reading Comprehension and New Word Learning

Reading comprehension and the ability to learn the meanings of new words are related: less skilled comprehenders have difficulties acquiring new word meanings from the reading context (Cain et al., 2004b) as well as through explicit instruction (Nation et al., 2007), and learning new words through the use of definitions and sentence contexts predicts reading comprehension in adults (Bolger et al., 2008). In the current study sentence contexts and definitions are used to facilitate new word learning. To the extent that processing these definitions and sentence contexts requires previously acquired comprehension skills and verbal knowledge, we expect reading comprehension level and new word learning to also be related in this study, akin to a Matthew Effect (Stanovich, 1986) for vocabulary acquisition (Lawrence et al., 2016). However, in terms of providing new contributions to the literature, we are most interested in whether reading comprehension level interacts with instructional and morphological variables to predict new word learning.

Current Study

This study had five aims: First, to test whether new word learning is better when meaning is reinforced across different sentence contexts (semantic reinforcement condition) or when it is discriminated through the use of highly alignable example and non-example sentence contexts (semantic discrimination condition); second, to test whether one aspect of morphology - affix frequency, affects new word learning in adolescent readers; third, to test whether child characteristics – reading comprehension level and working memory contribute to new word learning and interact with learning condition and morphological information; fourth, to determine whether effects of the two learning conditions vary across 6^th to 12^th grades and across readers of different skill levels within those grades; and fifth, to determine whether the ability to learn new words under different learning conditions contributes uniquely to the prediction of reading comprehension, over and above other known determinants of reading comprehension, such as word reading efficiency, previously acquired vocabulary (Lawrence et al., 2016), and working memory (Cain et al., 2004b).

To accomplish these aims, we tested a large number of students between 6^th and 12^th grades, oversampling for less skilled comprehenders. This large sample enabled us to take a continuous rather than a categorical approach to analysis by treating reading comprehension continuously. Two different contextual learning conditions were manipulated within-subject and the effects of affix frequency on learning were also tested within-subjects.

In addition to the use of a continuously distributed reading comprehension score to study individual differences in new word learning, we also employed explanatory item response models in order to estimate outcomes as a function of both reader and item characteristics. We used these models because we wanted to know whether there are moderating effects of item characteristics (affix frequency and learning condition) on the relation of reader characteristics (grade, comprehension level, and working memory) to new word learning. Although this approach has not been frequently implemented in literacy studies, it is advantageous as it allows simultaneous investigation of interactive effects of reader and item characteristics on test performance. In other words, through explanatory item response models, it is possible to use reader characteristics along with item characteristics to explain individual differences in responses to test items through their joint influences on ability and item difficulty.

We predicted that: 1) the discrimination learning condition might encourage more effortful processing and result in better learning; we had no basis on which to predict moderation by comprehension skill level or working memory based on the prior literature; 2) students would make use of higher frequency affixes to help them learn and remember new words, but more skilled comprehenders might be better able to take advantage of this morphological information; 3) because we found differences between middle school and high school students in this same sample on some text-level skills ((Barth, Barnes, Francis, Vaughn, & York, 2015), students in higher grades would outperform those in the lower grades; and 4) the ability to learn new words would account for additional variance in reading comprehension controlling for acquired vocabulary, word reading fluency, and working memory.

Method

Participants

School sites.

Schools from three cities and one suburb within a larger metropolitan area in the south central United States took part in the study. There were five middle schools (serving grades 6–8) and five high schools (serving grades 9–12).

Screening criteria.

In order to obtain roughly equal and large numbers of adequate and struggling comprehenders with adequate word reading skills, two levels of screening were used. For the initial screening, the Texas Assessment of Knowledge and Skills English Language Arts Assessment (TAKS-ELA; Texas Education Agency, 2003) from the previous school year was used to identify students likely to fall into struggling and adequate comprehender groups.

On this group-administered test, students read expository and narrative passages and answered questions designed to assess comprehension of the literal meaning of the passages, understanding of critical vocabulary, and ability to reason using information in the text. Reliability coefficients for grades 6–11 ranged from .73 to .89.

Students with TAKS-ELA scaled scores at or below 2,150, were identified as struggling readers, which included students below the cutoff of 2,100 and those who had a scaled score whose lower bound 95% confidence interval included a failing score (i.e., between 1,200–2,150). Students scoring at least one standard error of measurement above the passing score (i.e. higher than 2,150) were identified as adequate comprehenders. Students were excluded from participation only if they were currently identified as Limited English Proficient or their English Language Arts instruction was not provided in a general education setting, or they had an identified intellectual disability, severe behavioral disorder, or autism. Of all the students who qualified for participation based on these primary inclusion and exclusion criteria, 25–50 students from each grade in each school were randomly selected for participation with roughly equal numbers selected from each of the TAKS-ELA groups (i.e., struggling vs. adequate comprehenders) at each grade.

Of the 1,929 students who were approached for participation, 166 refused consent. In the second level of screening, the remaining 1,763 students were screened for adequate word reading ability based on the Woodcock Johnson-III Letter Word Identification subtest (WJ-III LWID, Woodcock et al., 2007). Students with scores below the 20th percentile for their grade on the WJ-III LWID were excluded resulting in the loss of 411 students. Eighty percent of those who were excluded due to low word reading ability were from the group that failed TAKS-ELA.

Selection of students for struggling and adequate comprehender groups.

The remaining 1,352 students (694 male and 658 female), were administered a standardized assessment of reading comprehension, the Gates-MacGinitie Reading Comprehension Test (GMRT-RC; MacGinitie et al., 2000) to measure reading comprehension skill. There were 107 students who chose not to take the GMRT-RC and five GMRT-RC assessments were excluded due to reliability issues. This resulted in a sample of 1,240 students, 693 adequate comprehenders (GMRT-RC scores above the 25^th percentile) and 547 struggling comprehenders (GMRT-RC scores at or below the 25^th percentile). Note that the information concerning comprehender groups based on the GMRT-RC is merely used to describe the sample; a continuous reading comprehension score was used in statistical analyses.

Materials and Procedures

A subset of measures from a larger assessment battery, including reading achievement measures and an experimental measure of new vocabulary learning, was selected for analysis in the current study. All measures were individually administered, unless stated otherwise.

WJ-III Letter-Word Identification (WJ-III LWID, Woodcock et al., 2007) assesses the ability to read real words without time constraints. The WJ-III LWID grade based standard score was used in the current study for screening purposes as described above. Within-grade internal consistency reliabilities for this sample ranged from .89 to .92.

WJ-III Working Memory (WJ-III NR; Woodcock et al., 2007) assesses verbal working memory. The task consisted of seven levels including trials with strings of digits of variable length. Immediately after the oral presentation of a digit string, the student was required to recall and repeat the string of digits in reverse order. The WJ-III NR grade based standard score was used in the current study. Within-grade internal consistency reliabilities for the sample were 0.75 to 0.83.

Test of Word Reading Efficiency (TOWRE; Torgesen et al., 1999) includes two subtests. On Sight Word Efficiency a participant is given a list of words and asked to read them as accurately and as quickly as possible within 45 seconds. For Phonemic Decoding Efficiency, a student is given a list of non-words and is asked to read them as accurately and as quickly as possible within 45 seconds. A standard composite score of Sight Word Efficiency and Phonemic Decoding Efficiency was used in the current study. According to the TOWRE manual, alternate forms reliabilities range from .91 to .97 depending on age.

GMRT-Vocabulary (GMRT-V; MacGinitie et al., 2000) is a group-administered test used to measure preexisting knowledge of common and rare words. Students are presented with words placed within a context suggesting a part of speech of the word (but without other context clues), and then are asked to select the most appropriate word or phrase related to the target word. The GMRT-V extended scale score was used in the current study. Within sample internal consistency reliabilities ranged from .83 to .90.

GMRT-Reading Comprehension (GMRT-RC; MacGinitie et al., 2000) is group administered to assess passage-level reading comprehension. Students read passages and then answer multiple-choice questions. Providing correct answers to questions depends on the ability to recall information, which is provided in the passage, and the ability to draw inferences. The GMRT-RC extended scale score was used in the current study. Three forms were used: 6^th grade form S; 7–9^th grade form S; and 10–12^th grade form S. Within sample internal consistency reliabilities ranged from .89 to .94.

Test of Silent Reading Efficiency and Comprehension (TOSREC; Wagner et al., 2010). Students are given 3 minutes to read and verify correctness of as many sentences as possible. Form O was used in the current study. The TOSREC index score was used in the current study to assess sentence-level reading fluency and comprehension. According to the TOSREC manual, alternate-form coefficients range from .86 to .93 depending on the grade.

New Vocabulary Learning (NVL) Task consisted of an instructional phase, where participants were provided with the meaning of new words as well as exemplars of those words in sentence contexts, with half of the words taught in a semantic reinforcement condition and half in a semantic discrimination condition, with three learning tests in which the participant: (1) provided the definition for the new word immediately after the word was taught in the instructional phase (Open Response Word Definition measure); (2) selected the new word from 3 foils when given the word’s definition immediately following the instructional phase (Multiple Choice Word Recognition measure); and (3) judged whether the new words were used correctly in sentences 24 – 72 hours later (Judging Sentence Meaning measure).

Instructional phase.

Words were taught in the same session with half of the learning trials presented in a Semantic Reinforcement Learning (SRL) block and the other half in a Semantic Discrimination Learning (SDL) block such that learning condition was manipulated within subjects. SRL and SDL conditions and their order of presentation were counterbalanced across participants so that every word appeared equally often in each learning condition across subjects, and so that SDL and SRL trials were equally often presented in Blocks 1 and 2. A practice trial was administered at the beginning of each block of learning trials. For each word, the examiner said aloud the target word which was also shown in print. The student repeated the word, and then the examiner said aloud the definition. Co-presentation of the orthographic, phonological and semantic aspects of the words was based on findings showing better retention of new vocabulary when the spelling of the word is included with its pronunciation and meaning (Rosenthal & Ehri, 2008). Next, the student read aloud the two sentences. After each sentence, the examiner provided a scripted explanation of why the sentence was or was not a correct example of the word meaning, depending on the learning condition.

Word stimuli.

Fourteen rare words were used (12 test words and 2 practice words in Appendix A). These words were a combination of nouns, verbs, and adjectives. In the Educators Word Frequency Guide (1995) norms, twelve of the rare words did not occur and two rare words had a frequency count of less than one per million. We chose these words because: 1) they were all unknown to skilled 12^th grade comprehenders based on a pilot study; and 2) despite being matched for word frequency, the words varied in affix frequency. Our interest was not to teach students new words per se, but to use the word set to test two types of word learning and the role of affix frequency in that learning.

All the words were derived words because inflected forms (i.e. –s, -es, -ed, and –ing) are mastered in the earlier grades. Furthermore, the roots were bound morphemes (i.e. carry meaning only when bounded to another morpheme) to control for the possibility that a derived word’s meaning and spelling pattern might be remembered better because the root morpheme was a real word and, therefore, had meaning, rather than the affix to which it was bound.

The affixes varied in frequency (low, high, none) to explore whether learning of new word meaning was better for words with high frequency affixes, low frequency affixes, or no affixes. Affix frequency was based on the MRC Psycholinguistic Database, which is an online computer database that contains 150837 words and provides information about 26 different linguistic properties of those words. Four words were chosen for each affix condition. In words with affixes, the four high frequency affixes were -ish, -ive, con- and pro- and the four low frequency affixes were ecto-, e-, -tic and -tus. There were two prefixes and two suffixes in both affix conditions.

The words were chosen so that all graphemes corresponded with clearly pronounced phonemes. This controlled for the possibility that two-morphemic words were more difficult than one-morpheme words because of the presence of neutral sounds in polysyllabic words, such as ‘schwa’ vowels.

Construction of word definitions and sentences.

Procedures outlined by Markle and Tiemann (1969) were used to construct the definitions and sentences for the learning trials so that definitions specified the critical attributes associated with the word meaning. For example, the definition of ‘conclave’ specified two critical attributes, which were private or secret and meeting, which were incorporated into the definition (i.e., students were taught that ‘conclave means a private or secret meeting’).

The first sentence read by the student was the same regardless of learning condition. This first sentence reinforced the new word’s meaning by presenting it in a semantically correct context. The example referred to all the critical attributes identified in the definition, so that the word meaning was contextualized and non-essential attributes associated with the word meaning were identified. For example, for the word ‘conclave’, the people involved in the meeting, the location of the meeting, and purpose of the meeting, were identified as non-essential attributes and the critical attributes of ‘private’ and ‘meeting’ were used to construct the sentence; ‘The football team held a conclave in the locker room to discuss the game.’

In the SRL condition, the second sentence was constructed to reinforce the word’s meaning in a different semantic context from the context used in the first sentence and the same non-essential attributes used to produce the first sentence were varied to produce this second sentence. For example, the second sentence for ‘conclave’ was; ‘The robbers arranged a conclave deep in the forest to plot the next robbery.’

In the SDL condition, the second sentence provided a non-example of the word’s meaning. To construct this discrimination sentence only one critical attribute was changed. For example, for ‘conclave’ the idea of a ‘private’ meeting was changed to a ‘public’ meeting in the sentence ‘The football team held a conclave on national television to discuss the game’. None of the non-essential attributes were changed, so these non-essential contextual features were similar across all sentence stimuli. Thus, the two sentence examples in the SDL condition were highly similar, but differed in one attribute critical to the new word’s meaning.

Open response word definition measure.

After the completion of each learning trial for each word (examiner-read definition and student-read sentences), the examiner asked the student to provide a definition for the word. This assesses the student’s ability to use both the definition and the context sentences to provide the meaning of the word immediately after learning. The examiner wrote down the student’s answer verbatim and provided one non-directive probe per trial when partial definitions were given and then moved on to the next learning trial. Student definitions that referred to all critical attributes of the new word’s meaning and indicated accurate knowledge of the word’s meaning were considered as correct responses and were given a score of 1. Answers were scored in the field and then double-checked in the lab using the scoring manual created for this study and based on piloting of the task. Disagreements typically involved answers that did not appear in the manual. These were solved through discussion followed by updating of the scoring manual. A more detailed two-point scoring system that awarded a score for a partially correct answer produced the same findings. The analyses reported below used the binary scoring system.

Multiple choice word recognition measure.

Immediately after completing all of the trials in the instructional phase and the open response definition measure, participants were tested individually on their ability to recognize the newly learned words through meaning using a multiple-choice test. Students were required to match a definition with a word from 4 choices: the correct word (e.g., conclave), one of the other newly learned words (e.g., rictus), and two non-words that were orthographically/phonologically similar to the two learned words but differed by one grapheme and corresponding phoneme (e.g., donclave, rictun). The non-words were constructed to assess students’ ability to recognize newly learned word forms (orthographic and phonological) through meaning in an attempt to assess the lexical quality of newly learned words (Perfetti, 2007). In words that contained an affix, the changed grapheme/phoneme was always part of the affix (e.g., ‘c’ was changed to’d’ in ‘con-’ to produce the non-word ‘donclave’). The order of the word choices was randomized for each definition. The examiner read aloud the definition as the student read along on the student test sheet and then filled in the bubble with their answer choice. We used this task to tap lexical quality; that is, whether students connected orthography, phonology, and meaning to discriminate between newly learned words. We also looked at whether errors were based on a partially consolidated orthographic-phonological representation (i.e., choosing the word that was orthographically and phonologically similar to the correct word) or on not having integrated orthography and phonology with meaning (i.e., choosing a wrong, but newly learned word).

Judging sentence meaning measure.

Twenty-four to 72 hours later students were tested (792 after 24 hours, 287 after 48 hours, and 60 after 72 hours) in small groups on their ability to judge whether each newly learned word was being used correctly in two different sentence contexts; in one sentence the word was used correctly (True sentence) and in another sentence the word was used incorrectly (False sentence). They were reminded that this task would require them to make judgements on the new words they had learned one to three days previously. A sentence was read aloud by the examiner as students read along on their student test sheet and they marked whether the sentence was true or false before the next sentence was read. Research staff served as monitors to ensure compliance and help with pacing. The test sentences were presented in a fixed random order. This task was used to measure longer term retention of the newly learned words and their meanings.

The same non-essential attributes used to create the context sentences in the learning trials, were used and varied to produce the true sentence for this measure. In both the true and false sentences, the new word occupied the same part of speech (e.g., conclave was used as a noun in both sentences) and both sentences used approximately the same number of words. For example, the true sentence for ‘conclave’ was: The club members had a conclave in the conference room to elect a new president; and the false sentence for ‘conclave’ was: The old conclave in the back yard needed a new paint job’. Neither of the sentences had been used in the learning trials.

Internal consistency was calculated using Kuder-Richardson 20 (KR-20) for all NVL subtests. Within sample internal consistency reliabilities for grades 6 through 12 ranged from .42 to .77 for the Open Response Definition measure; from .54 to .68 for the Multiple Choice Word Recognition measure; and from .67 to .83 for the Judging Sentence Meaning measure.

Data Analysis

The statistical analyses were performed using the PROC GLIMMIX and PROC REG procedures in SAS 9.4 software (SAS, Inc., Cary, NC). We used binary, cross-classified random effects, explanatory item response (EIRT) models to examine a probability of correct responses to NVL items as a function of reader characteristics and item characteristics. EIRT models are special cases of generalized linear or nonlinear mixed models. The models jointly explain item responses on a test in terms of: (a) the effects of reader characteristics on a latent ability (θ_p – in our case, a reader’s location on a NVL continuum), and (b) the effects of item characteristics on item difficulty (β_i - difficulty of an item desgined to measure the latent ability; De Boeck & Wilson, 2004). These models are particularly advantageous when one is interested in investigating moderating effects of item characteristics on relations between reader characteristics and readers performance on an outcome measure (reader-item interactions). The interactions provide unique insights about how the same item characteristic differently affects readers depending on their individual characteristics. Importantly, these insights cannot be easily examined when looking at interaction effects based on composite scores.

In the estimated EIRT models, reader characteristics were used to explain variability in ability to learn new words, while item characteristics were used to explain variability in item difficulty (De Boeck & Wilson, 2004). Responses to NVL items were scored as correct-incorrect, with missing values coded as incorrect responses. Reader characteristics included grade and continuously measured reading comprehension and working memory scores while item characteristics included learning conditions (SRL and SDL) and affix frequency (none, low frequency, and high frequency).

For each NVL measure, four classes of models were estimated (1 – 4 below). The variance reduction estimates were used to differentiate between estimated models (Molenberghs & Verbeke, 2007).

1. Unconditional Model without predictors was used to estimate unconditional variances in ability to learn new words and item difficulty.

2. Main Effects Model of Item Characteristics included item characteristics as predictors, and was estimated to determine whether learning condition and affix frequency significantly predict item difficulty. The Fisher’s LSD was used as a correction method for post-hoc comparisons involving affix frequency as this method is the most powerful approach when comparing three groups (Levin et al., 1994). For ease of interpretation, the log of odds of statistically significant comparisons were converted to a standardized mean difference d (Hasselblad & Hedges, 1995):

   $d=\mathit{log}\mathit{\text{OddsRatio}}*\frac{\sqrt{3}}{\pi },$ where π is the mathematical constant (approximately 3.14).

3. Main Effects Model of Reader Characteristics included reader characteristics as predictors, and was estimated to determine whether grade, working memory, and reading comprehension significantly predict learning new words.

4. Interactive Effects Model included reader characteristics, item characteristics, and interactions between reader and item characteristics, and was estimated to determine whether relations between reader characteristics and learning new words differ depending on learning conditions and/or affix frequency.

Results

The final sample used in analyses included 1,214 students who completed at least one of the three NVL measures. Twenty-five out of 1,240 students were excluded because they declined to fill out at least one of three NVL measures, and/or due to reliability issues related to the NVL measures. Forty nine percent of participants were female; 65% were eligible for free or reduced meals under the National School Lunch and Child Nutrition program; 51% were Hispanic, 23% were White, 21% were African-American, and 5% identified themselves as an “other” category of ethnicity. No students were currently identified as English Language Learners. Table 1 presents mean scores for reading achievement measures by comprehender group. Table 2 presents mean scores for NVL measures by grade.

Table 1.

Mean Scores (Standard Deviation) on Reading Achievement Measures by Comprehender Group Chosen on the Basis of TAKS-ELA Screening

	Adequate Comprehenders		Struggling Comprehenders
Reading Achievement Measures	n	Mean (SD)	n	Mean (SD)
WJ-III LWID (Grade-based standard score)	677	102.22 (9.48)	538	96.50 (6.47)
GMRT-RC Lexile	677	1016.54 (132.34)	538	759.72 (116.21)
TAKS-ELA Lexile	677	1161.77 (225.49)	538	972.99 (205.53)
TOSREC (Index score)	664	97.39 (11.15)	531	86.84 (9.83)

Note. WJ-III LWID = Letter-Word Identification subtest of the Woodcock & Johnson Tests of Academic Achievement; GMRT-RC = Gates-MacGinitie Reading Tests - Reading Comprehension test; TAKS = Texas Assessment of Knowledge and Skills-English Language Arts; TOSREC = Test of Silent Reading Efficiency and Comprehension.

Table 2.

Mean Scores (Standard Deviation) on New Vocabulary Learning by Grade

	Open Response Word Definitions			Multiple-Choice Word Recognition			Judging Sentence Meaning
	n	Mean (SD)	Min-Max	n	Mean (SD)	Min-Max	n	Mean (SD)	Min-Max
Grade 6	121	8.03 (2.36)	2–12	121	7.93(2.36)	3–12	120	14.58 (4.35)	7–24
Grade 7	174	8.22 (2.33)	1–12	173	8.41 (2.30)	2–12	164	13.75 (4.25)	5–24
Grade 8	187	8.61 (2.37)	0–12	187	8.49 (2.23)	2–12	180	14.52 (4.83)	4–24
Grade 9	191	8.91 (2.35)	1–12	178	8.54 (2.41)	2–12	178	14.87 (4.59)	6–24
Grade 10	206	9.67 (2.00)	3–12	205	9.52 (2.07)	4–12	189	15.59 (5.11)	5–24
Grade 11	193	9.70 (1.96)	0–12	190	9.23 (2.23)	4–12	180	15.73 (5.06)	6–12
Grade 12	142	9.89 (1.69)	4–12	141	9.65 (2.06)	3–12	131	16.06 (5.04)	8–12
Total	1214	9.05 (2.26)	0–12	1195	8.86 (2.30)	2–12	1142	15.01 (4.82)	4–24

Effects of Learning Conditions and Affix Frequency on New Vocabulary Learning

Table 3 contains log of odds with corresponding standard errors, and probability of correct responses to the open response word definitions items, multiple choice word recognition items, and judging sentence meaning items estimated in models with learning conditions and affix frequency as predictors (Main Effects Models of Item Characteristics). We converted model estimates (i.e., log of odds) to expected probabilities of correct responses to facilitate discussion of the results.

Table 3.

Log of Odds and Estimated Probabilities of Correct Responses Estimated in Main Effects Models of Item Characteristics by New Vocabulary Learning Measures

Open Response Word Definitions*†
	Log of Odds	Std. Error	Probability of Correct Responses
SRL condition	1.62	0.25	0.62
SDL condition	1.46	0.25	0.59
No affix	2.07	0.43	0.67
Low frequency affix	0.02	0.31	0.02
High frequency affix	2.52	0.31	0.72
Multiple-Choice Word Recognition*†
	Log of Odds	Sth. Error	Probability of Correct Responses
SRL condition	1.23	0.14	0.55
SDL condition	1.31	0.14	0.57
No affix	1.38	0.14	0.58
Low frequency affix	1.19	0.14	0.54
High frequency affix	1.23	0.14	0.55
Judging Sentence Meaning†
	Log of Odds	Sth. Error	Probability of Correct Responses
SRL condition	1.45	0.09	0.59
SDL condition	1.44	0.09	0.59
No affix	1.34	0.09	0.57
Low frequency affix	1.44	0.10	0.59
High frequency affix	1.56	0.09	0.61

Note. SRL = Semantic Reinforcement Learning; SDL = Semantic Discrimination Learning;

^*main effect of learning condition statistically significant;

^†main effect of affix type statistically significant.

Both, learning condition and affix frequency significantly predicted the probability of correct responses to the open response word definition items (F(1, 1138) = 13.23, p < .001, and F(2, 1138) = 422.05, p < .001, respectively), and multiple choice word recognition items (F(1, 1130) = 3.88, p = .049, and F(2, 1130) = 5.89, p < .01, respectively). Affix frequency, F(1, 1112) = 9.89, p < .001, but not learning conditions, F(2, 1112) = 0.11, p = .745, was a statistically significant predictor of correct responses to the judging sentence meaning items.

While new word learning was better under the SDL condition for multiple choice word recognition items (d = 0.04), the SRL condition was associated with better performance for the open response word definitions items (d = 0.09), though the differences in mean probabilities between the two conditions were small in both instances. For the open response word definitions measure, items with a low frequency affix were significantly harder relative to items with no affix (d = 1.13, p < .001), or those with a high frequency affix (d = 1.38, p < .001). For the multiple choice word recognition measure, items with no affix were significantly easier relative to items with a low frequency affix (d = 0.11, p = .001), or a high frequency affix (d = 0.08, p = .008), though the magnitude of differences was small in both instances. For the judging sentence meaning measure, items with no affix were significantly harder relative to items with high frequency affix (d = 0.12, p < .008).

Effects of Reader Characteristics on New Vocabulary Learning

Table 4 includes significance tests for coefficients estimated in the Main Effects Models of Reader Characteristics. Both, grade and reading comprehension significantly predicted the probability of correct responses to the open response word definition items (F(7, 1112) = 30.16, p < .001, and F(1, 1112) = 152.95, p < .001, respectively), multiple choice word recognition items (F(7, 1106) = 2.28, p < .001, and F(1, 1106) = 102.50, p < .001, respectively) and the judging sentence meaning items (F(7, 1096) = 43.76, p < .001, and F(1, 1096) = 269.10, p < .001, respectively). For the open response definition and judging sentence meaning measures, performance was generally more accurate in the higher grades (10–12) than in the lower grades (6–8). There was no consistent pattern for grade when examining adjusted mean scores for the multiple choice word recognition items. Working memory significantly predicted the probability of correct responses to the open response word definition items (F(1, 1112) = 23.57, p < .001) and multiple choice word recognition items (F(1, 1106) = 20.50, p < .001), but not to judging sentence meaning items.

Table 4.

Significance Tests for Coefficients Estimated in Main Effects Models of Reader Characteristics by New Vocabulary Learning Measures

	Open Response Word Definitions		Multiple-Choice Word Recognition		Judging Sentence Meaning
	F-value	p-value	F-value	p-value	F-value	p-value
Grade	30.16	<.001	22.28	<.001	43.76	<.001
Reading comprehension	152.95	<.001	102.50	<.001	269.10	<.001
Working memory	23.57	<.001	20.50	<.001	3.23	0.073

Effects of Learning Condition and Affix Frequency on Relations between Reader Characteristics and New Vocabulary Learning

Table 5 includes significance tests for coefficients estimated in the Interactive Effects Models. Results suggested a main effect of reading comprehension on the open response word definitions, F(1, 1106) = 145.84, p < .001, and a main effect of working memory on the multiple choice word recognition, F(1, 1110) = 19.39, p < .001. These main effects are discussed here because they were not involved in any statistically significant interaction. Students with higher reading comprehension scores performed better on the open response word definitions measure, while students with higher working memory scores performed better on the multiple-choice word recognition measure.

Table 5.

Significance Tests for Coefficients Estimated in Interactive Effects Models by New Vocabulary Learning Measures

	Open Response Word Definitions		Multiple-Choice Word Recognition		Judging Sentence Meaning
	F-value	p-value	F-value	p-value	F-value	p-value
Learning condition	0.49	0.486	4.20	0.040	0.30	0.585
Affix frequency	3.50	0.030	2.15	0.117	1.08	0.339
Grade	1.94	0.070	1.22	0.294	2.40	0.026
Reading comprehension	145.84	<0.001	107.90	<0.001	282.48	<0.001
Working memory	23.54	<0.001	19.39	<0.001	3.20	0.074
Learning condition*grade	0.18	0.983	1.06	0.383	0.90	0.495
Reading comprehension*grade	2.08	0.052	1.54	0.162	2.70	0.013
Learning condition*reading comprehension	0.81	0.369	5.22	0.022	1.45	0.228
Learning condition*reading comprehension*grade	0.34	0.916	1.31	0.249	1.45	0.190
Affix frequency*reading comprehension	1.78	0.169	0.70	0.496	0.43	0.648
Affix frequency*grade	0.50	0.917	0.44	0.950	0.36	0.978
Affix frequency*reading comprehension*grade	0.63	0.822	0.45	0.945	0.31	0.989
Working memory*grade	2.76	0.011	0.86	0.525	0.47	0.832
Working memory*Affix frequency	1.33	0.264	2.32	0.099	0.62	0.535
Working memory*Affix frequency*grade	1.09	0.361	1.31	0.203	0.67	0.778
Working memory*Learning condition	1.05	0.306	0.26	0.613	3.11	0.078
Working memory*Learning condition*grade	1.45	0.193	0.98	0.436	1.23	0.287

Note. Statistically significant interaction effects are bolded.

Results indicated that affix frequency did not differentially affect ability to learn new words in comprehenders with different skill levels in grades 6^th through 12^th for the three NVL measures. Results also suggested that learning condition did not differentially affect the ability to learn new words in comprehenders with different skill levels in grades 6^th through 12^th for the open response word definitions and judging sentence meaning measures. In contrast, for the multiple choice word recognition measure, the effects of learning condition on learning new words varied across comprehenders with different skill levels, F(1, 1100) = 5.22, p = .022.

As depicted in Figure 1, for students at lower levels of reading comprehension the SRL and SDL conditions exerted similar influence, in so far as the estimated probability of correct responses was similar for items learned under the SRL and SDL conditions. In contrast, students at higher levels of reading comprehension seemed to benefit more from the SDL condition as the estimated probability of correct responses was higher for items learned under the SDL condition relative to the SRL condition for these students.

Figure 1.

A line plot demonstrating the probability of correct responses to Multiple Choice Word Recognition items for learning condition and reading comprehension. SRL = Semantic Reinforcement Learning; SDL = Semantic Discrimination Learning.

Although interactive effects of reading comprehension skills with grade level were not of primary interest, there was a statistically significant effect of reading comprehension and grade on the judging sentence meaning performance, F(6, 1093) = 2.70, p = .013. Sixth and 7^th graders with lower levels of reading comprehension had the lowest probability of correct responses to the judging sentence meaning items relative to students in other grades, whereas 10^th graders with higher levels of reading comprehension had the highest probability of correct responses to the judging sentence meaning items when compared to students in remaining grades.

Similarly, interactive effects of working memory with grade level were not of primary interest though there was a statistically significant effect of working memory and grade on the open response word definitions performance, F(6, 1106) = 2.76, p = .011. Seventh and 9^th graders with lower levels of working memory had the lowest probability of correct responses to the open response word definitions items relative to students in other grades, whereas 9^th graders with higher levels of working memory had the highest probability of correct responses to the open response word definitions items when compared to students in remaining grades.

Learning New Vocabulary Measures as Predictors of Reading Comprehension

A comparison of adjusted R² indices computed across different multiple regression models suggested that Model 1 including the word reading fluency measure and grade level explained 31% of variance on the GMRT-PC and 22% of variance on the TOSREC. By adding the three NVL measures to this model (Model 2) the proportion of variance explained increased to 44% and 39%, respectively, an increase that was statistically significant for both outcomes, (F(95, 1114) = 3.69; p < .001; F(92, 1098) = 4.16; p < .001, respectively). Model 3 with word reading and standardized vocabulary measures, controlling for grade level, explained 59% of the variance on the the GMRT-PC and 43% of the variance on the TOSREC. Adding the three NVL measures to this model (Model 4), increased the proportion of variance explained to 60% and 47%, respectively, an increase that was statistically significant for the TOSREC (F(92, 1095) = 1.67; p < .001), but not for the GMRT-PC (F(95, 1110) = 1.15; p = .16). In Models 5 and 6, after adding working memory to Models 3 and 4 the proportion of variance explained for both comprehension measures remained unchanged (Table 6).

Table 6.

Relations across Different Sets of Predictors with Text-Level Reading Comprehension and Sentence-Level Reading Comprehension Measures broken down by Six Models

		GMRT-PC		TOSREC
		β	p-value	β	p-value
Model 1	Word reading	0.29	<.001	0.49	<.001
	Grade	0.58	<.001	0.23	<.001
Model 2	Word reading	0.19	<.001	0.37	<.001
	Grade	0.44	<.001	0.04	0.098
	ORD	0.23	<.001	0.25	<.001
	MCD	0.16	<.001	0.26	<.001
	JSM	0.12	<.001	0.07	0.007
Model 3	Word reading	0.09	<.001	0.32	<.001
	Vocabulary	0.66	<.001	0.57	<.001
	Grade	0.17	<.001	−0.16	<.001
Model 4	Word reading	0.09	<.001	0.30	<.001
	Vocabulary	0.60	<.001	0.42	<.001
	Grade	0.17	<.001	−0.14	<.001
	ORD	0.07	0.002	0.13	<.001
	MCD	0.02	0.447	0.16	<.001
	JSM	0.05	0.022	0.01	0.533
Model 5	Word reading	0.07	0.001	0.30	<.001
	Vocabulary	0.65	<.001	0.55	<.001
	Working memory	0.09	<.001	0.09	0.001
	Grade	0.18	<.001	−0.11	<.001
Model 6	Word reading	0.07	0.002	0.29	<.001
	Vocabulary	0.60	<.001	0.42	<.001
	Working memory	0.08	0.001	0.06	0.019
	Grade	0.18	<.001	−0.13	<.001
	ORD	0.06	0.006	0.13	<.001
	MCD	0.01	0.640	0.15	<.001
	JSM	0.04	0.026	0.01	0.574

Note. ORD = Open Response Word Definitions; MCD = Multiple-Choice Word Recognition; JSM = Judging Sentence Meaning; GMRT-PC = Gates MacGinitie Reading Test Passage Comprehension; TOSREC = Test of Silent Reading Efficiency and Comprehension

Discussion

Consistent with vocabulary learning studies in younger children and adults with difficulties in reading comprehension (e.g., Cain et al., 2004b; Nation et al., 2007; Perfetti et al., 2005), less skilled adolescent readers had difficulty learning the meanings of new words compared to their more skilled peers. There are several factors, however, that distinguish this study from previous vocabulary learning studies. They include: 1) the use of explanatory items response methods to predict vocabulary learning by taking both student- and item characteristics and their interactions into account; 2) the preservation of continuity when examining individual differences in reading comprehension (i.e., treating reading comprehension as a continuous variable); 3) the within-subjects comparison of two different theoretically motivated learning conditions; 4) the investigation of the role of affix frequency on vocabulary learning; and 5) the focus on a large sample of adolescent readers across both the middle and high school years, a time span considered to be crucial for the teaching of academic vocabulary. This study is also distinct from other vocabulary learning studies (e.g., Bolger et al., 2008) because of the stringent way in which the added value of the vocabulary learning measures for predicting reading comprehension was assessed by including word reading efficiency, working memory, and already acquired vocabulary knowledge in the models.

Effects of Learning Condition on New Word Learning

Based on structural alignment theory, we predicted that semantic discrimination learning would be associated with better learning outcomes. However, we found this to be the case only for more skilled comprehenders when they were tested after all of the words had been instructed. In contrast, the semantic reinforcement condition resulted in higher immediate recall of new word meanings across comprehension skill level, with better performance associated with higher levels of comprehension skill. There were no effects of previous learning condition on the ability to judge whether sentences were semantically true or false 24–72 hours after initial learning; however, performance on this task was associated with comprehension skill level.

One way to think about the findings is to consider what is required in each learning condition and in relation to what more and less skilled comprehenders bring to the learning situation. In order to make maximal use of discrimination learning, the two context sentences must be compared with the non-example introducing semantic interference during this comparison process possibly making learning more difficult than in the semantic reinforcement condition (Bjork & Kroll, 2015). Learning that is more repetitive and less effortful is associated with better immediate recall and worse long term retention; learning that is more difficult and effortful is often associated with worse immediate recall and better longer-term retention (reviewed in Dunlosky et al., 2013). Although this framework fits the findings for the open response definition and multiple-choice tasks it does not explain why there would be no effects of learning condition on the delayed semantic judgement task.

Another way to think about the findings is from a cognitive load perspective, which takes both the learning condition and the learner’s knowledge and cognitive resources into account. In cognitive load theory, the comparison process in the semantic discrimination condition requires a high element of interactivity (active comparison between the example and non-example). In contrast, semantic reinforcement may require less element interactivity because the two sentences can be read and understood without reference to each other. This comparison process in semantic discrimination learning, therefore, is more effortful and requires relatively more working memory resources (Evans & Stanovich, 2013; Sweller, 2010) than does semantic reinforcement learning. These working memory resources are better developed in skilled compared to less skilled comprehenders (Cain et al., 2004a; Carretti et al., 2009; Ricketts et al., 2007) such that more skilled comprehenders are better able to take advantage of semantic discrimination learning than their less skilled peers.

This cognitive resource explanation was tested in the current study by including working memory in the models and testing for interactions with learning condition and comprehension level. These interactions were not significant; however, working memory was associated with performance on both the open response definition and multiple-choice tasks though not the semantic judgment task. It is possible that the single measure of working memory we used (a digits reversed task) lacked sensitivity to discriminate between the two conditions because it does not adequately capture those aspects of working memory that might be needed for optimal learning. It would be of interest to investigate whether, for example, the comparative processes in semantic discrimination learning, which are assumed to produce semantic interference, draw on more specific cognitive processes such as cognitive inhibition (Arrington et al., 2014), and updating (Martin et al., 2020).

Implications.

Comprehension level was associated with the ability to learn new words likely reflecting the fact that comprehension skills (which need not be reading-specific, see Nation et al., 2007) were needed to process the definitions and context sentences used in new vocabulary instruction; however, the interaction of reader skill with learning condition suggests that vocabulary instruction for less and more skilled comprehenders might require differentiation not only with respect to instructional intensity, but also with respect to instructional techniques. Techniques such as the use of contextual variation and structural alignment that positively affect learning have largely been tested in typically achieving students. In the absence of evidence to the contrary, these learning techniques are assumed to operate similarly in individuals of different skill levels. The general principle is that varied contexts (like semantic reinforcement or semantic discrimination) affect encoding and retrieval strength. However, the findings in the current study suggest that these effects could be moderated by skill level. It would be of interest to test whether less skilled comprehenders benefit from additional practice with semantically correct exemplar sentences and retrieval practice that is distributed across time (Dunlosky et al., 2013), perhaps with non-examples used only later in instruction.

Effects of Affix Frequency on New Word Learning

We predicted that students would make use of higher frequency affixes to help them learn and remember new words, but that more skilled comprehenders might be better able to take advantage of morphological information than their less skilled peers given studies that show relations of morphological knowledge, vocabulary, and reading comprehension (e.g., Carlisle, 2007; Deacon & Kirby, 2004; Levesque et al., 2018). In this study, there was a variable pattern of findings that depended somewhat on the task used to measure new word learning. However, there were no interactions with grade, and, in contrast to our predictions, there were no interactions with comprehension skill level.

Depending on the task, there were advantages for words with no affixes (multiple-choice) or with high frequency affixes (judging sentence meaning) or both (open response definitions). Although word frequency was similar across affix conditions, words with no affix differed from high and low frequency affix words in word length and number of syllables. Multi-syllabic low frequency words are more difficult to read than single syllable low frequency words (Jared & Seidenberg, 1990), but children use their knowledge of morphological units to read unknown multi-syllabic words (Kearns, 2015). Our findings are compatible with the idea that syllable length and morphological frequency contribute to learning, memory, and semantic analysis of new words. We do not know why there was a somewhat different pattern of findings on the two tasks students completed during the single-session study. However, to the extent that performance on the delayed semantic judgement task may reflect forgetting over time (i.e., there were no effects of learning condition or of working memory), students may have relied on the only information available to them on this task– namely, their knowledge of high frequency affixes in their semantic analysis of the sentences.

Implications.

These findings, in combination with those from systematic reviews of morphology instruction (e.g., Bowers et al., 2010; Ford-Connors & Paratore, 2015), as well as observational studies (e.g., Silverman & Hartfrant, 2019) suggest that whole-class or small group approaches that provide explicit instruction in morphological analysis may be beneficial for both skilled and less skilled older readers and may allow less skilled older readers to take advantage of their relatively intact ability to use high frequency affixes to learn the meanings of new words.

Effects of Grade on New Word Learning

Given that we found differences between middle school and high school students in this same sample on some text-level skills (Barth, Barnes, Francis, Vaughn, & York, 2015), we predicted that students in the higher grades would outperform those in the lower grades. Although higher grades were associated with better performance on two of the measures in the main effects model, there were a few interactions with grade in the interactive effects model. Students in the two lowest grades (6 and 7) who were less skilled comprehenders were less accurate on the sentence verification task versus less skilled comprehenders in the higher grades and more skilled comprehenders overall. Because this task is done one or more days after initial learning when learning condition effects are no longer discernable, it may be that performance on this language-based task reflects a very low level of baseline performance for these youngest least skilled comprehenders.

Relations of New Word Learning to Reading Comprehension

Consistent with other studies of new vocabulary learning (e.g., Bolger et al., 2008; Cain et al., 2004b; Nation et al., 2007) findings from the explanatory item response models discussed above revealed a strong relation of reading comprehension level and new vocabulary learning. The regression analyses additionally tested whether the ability to learn new words would be related to reading comprehension even after controlling for several other comprehension-related factors including previously acquired vocabulary as well as grade, word reading efficiency and working memory (c.f. Bolger et al. 2008). The ability to learn new vocabulary was a significant and strong predictor of reading comprehension over and above word reading fluency and working memory, accounting for an additional 13 to 17 percent of the variance in both measures of reading comprehension. However, new vocabulary learning accounted for unique effects (4 percent) in the most stringent model containing acquired vocabulary knowledge, for only one measure of reading comprehension – the TOSREC.

Hypotheses about the relation of vocabulary and reading comprehension include those that are unidirectional (i.e., growth in vocabulary causes growth in reading comprehension or vice versa), bidirectional, or mediated by a third variable such as general verbal ability (Anderson & Freebody, 1981; Quinn et al., 2015; Verhoeven & van Leeuwe, 2008). The findings from our concurrent correlational models cannot be used to infer directionality or causality; however, given that the ability to learn new vocabulary contributed unique variance on one of the reading comprehension measures over and above previously acquired vocabulary knowledge, longitudinal studies conducted to understand the vocabulary-reading comprehension relationship might consider not only the measurement of acquired vocabulary but also whether and how mechanisms involved in new word learning are related to the development of reading comprehension. Furthermore, given the variable findings in this study for different measures of reading comprehension, combined with findings that different reading comprehension measures address somewhat different component skills of reading (e.g., Keenan et al., 2008), it may also be of interest to test whether a measure of new word learning predicts reading comprehension over and above these other predictors in a latent variable framework.

Limitations

Despite the strengths of the study described above, there are several limitations. First, comparisons across measures and across time are complicated by the fact that different measures were used at different time points. This makes it difficult to determine if some of the effects are related to features of the measures themselves (content, response format, reliability), to forgetting over time, or both. Second, our measures were primarily ones drawing on more explicit word knowledge – production of a word’s meaning, linking definitions with their orthographic forms, and judging the truth of sentences containing new words. Although explicit measures that require generation of definitions or recognition of words and their meanings have face validity in terms of what is required of students when learning new vocabulary, other measures may be more sensitive for tapping what has been learned. For example, priming paradigms that look at the association between new words and known words provide more implicit measures that can be sensitive indicators of the nature and quality of the representations of newly learned words. In this respect, we looked at errors on the multiple-choice task to provide a more sensitive assessment of the degree to there was binding of orthographic, phonological and semantic information, important for efficient new word learning (Ehri, 2014). Errors were most frequently due to choosing an incorrect new word (rictus instead of conclave; 15% of the time) compared to choosing an incorrect, but close spelling (donclave instead of conclave; 8% time), or an incorrect nonword (rictun; (3.5% of the time).

Third, because the new word learning task was only one of several tasks in a larger assessment study, we asked a fairly narrow range of questions about new vocabulary learning in skilled and less skills adolescent comprehenders. We chose to contrast two types of learning controlling for the number of experiences that each participant had for each word in each condition. Because greater numbers of exposures to a word in varied contexts are associated with greater learning (Bolger et al., 2008), it is difficult to generalize about the effectiveness of reinforcement versus discrimination learning beyond the definitions and two-exemplar format that we tested. However, we do note that the extent of instruction and exposure devoted to new vocabulary in school may not be much different from that used in the current study (Silverman & Hartranft, 2019). Fourth, due to time constraints, we also used a fewer number of words than is typically the case in vocabulary learning studies with college students, which might have had an effect on the sensitivity of our measures. Fifth, the manipulation of affix frequency tested few items per frequency condition, which might have also affected the reliability of the findings for this variable, possibly resulting in low sensitivity to differences due to affix frequency (i.e., increasing Type II error).

Summary and Implications

Given the overall lower performance of less skilled comprehenders regardless of learning condition, research on word learning interventions that narrow these gaps is critical. Optimal approaches for vocabulary instruction for struggling comprehenders may need to test different instructional methods as well as manipulate duration, intensity, and spacing of instruction in order to produce stronger representations linking orthography, phonology and semantics. As mentioned above, less skilled comprehenders might benefit from approaches, at least early in learning, that would not be equally advantageous to typically or high achieving adolescent readers. More generally, the findings suggest that the effect of general cognitive learning principles when applied to instructional contexts (e.g., structural alignment theory, varied exemplar based learning) may be moderated by student characteristics, in this case, reading comprehension level. In keeping with the analytic approach adopted in this study, we suggest that customization of vocabulary interventions may require attention to the reader’s knowledge and skills, to text or word characteristics, and to their interactions (Kintsch, 1986).

• The ability to learn the meaning of new words is particularly important for understanding informational texts in the secondary grades

• Both less and more skilled readers in grades 6 to 12 use what they know about suffixes and prefixes to learn and remember new words

• New vocabulary learning is facilitated in more skilled adolescent comprehenders by providing definitions followed by contrasting example and non-example sentences during learning

• Less skilled adolescent comprehenders do not show the same learning advantage for contrasting example and non-example sentences

Appendix A.

Design of Word Stimuli for the New Vocabulary Learning Measure

Word (non-word)	Affix frequency	Affix type	Affix	Number of morphemes	Number of syllables	Word class
Lithe (liche)				1	1	adjective
Screed (screel)				1	1	noun
Acrid (scrid)				1	2	adjective
Usurp (usurl)				1	2	verb
Churlish (churlich)	HF	suffix	-ish	2	2	adjective
Lenitive (lenitide)	HF	suffix	-[t]ive	2	3	adjective
Conclave (donclave)	HF	prefix	con-	2	2	noun
Proboscis (preboscis)	HF	prefix	pro-	2	3	noun
Rictus (rictun)	LF	suffix	-tus	2	2	noun
Noetic (noetim)	LF	suffix	-tic	2	3	adjective
Edict (odict)	LF	prefix	e-	2	2	noun
Ectopic (estopic)	LF	prefix	ecto-	2	3	adjective

Appendix B.

Example of Five Sentences Constructed to Teach and Test Student Learning of New Word ‘conclave’ in both Learning Conditions

	Learning condition
Sentence stimuli	Semantic reinforcement	Semantic discrimination
Learning trial
First sentence:	The football team held a conclave in the locker room to discuss the game.	The football team held a conclave in the locker room to discuss the game.
Second sentence:	The robbers arranged a conclave deep in the forest to plot the next robbery.	The football team held a conclave on national television to discuss the game.
Judging sentence meaning post-test
True sentence:	The club members had a conclave in the conference room to elect a new president.	The club members had a conclave in the conference room to elect a new president.
False sentence:	The old conclave in the back yard needed a new paint job.	The old conclave in the back yard needed a new paint job.