Evaluating the Impact of a Multistrategy Inference Intervention for Middle-Grade Struggling Readers

Abstract

Purpose

We examined the effectiveness of a multistrategy inference intervention designed to increase inference making and reading comprehension for middle-grade struggling readers.

Method

A total of 66 middle-grade struggling readers were randomized to treatment (n = 33) and comparison (n = 33) conditions. Students in the treatment group received explicit instruction in 4 inference strategies (i.e., clarification using text clues; activating and using prior knowledge; understanding character perspectives and author's purpose; answering inferential questions). In addition, narrative and informational texts were carefully chosen and sequenced to build requisite background knowledge to form inferences. Intervention was delivered in small groups of 3 students for 10 days of instruction.

Results

One-way analysis of covariance models on outcome measures with the respective pretest scores as a covariate revealed significant gains on a proximal measure of Egyptian-content knowledge (g = 1.37) and on a standardized measure of reading comprehension—i.e., Wechsler Individual Achievement Test–Third Edition Reading Comprehension (g = 0.46).

Conclusion

The moderate effect on a standardized measure of reading comprehension provides preliminary evidence for the effectiveness of this multistrategy inference intervention in improving reading comprehension of middle-grade struggling readers.

Although the United States has invested billions of dollars in teaching children how to read, many students advancing into middle school do not read at proficient levels (Aud et al., 2013). According to the 2013 National Assessment of Educational Progress, 91% of eighth-grade students with disabilities scored below the proficient level of performance on the reading test and 61% of eighth-grade students without a disability scored below the proficient level (Aud et al., 2013). This means that for many students, the instructional practices provided through current educational programming in the elementary grades do not sufficiently prepare them to read for understanding in the middle grades. As a consequence, this has led researchers, educators, and state and national educational agencies to ask: What does it mean to read for understanding in the middle grades (Mathis, 2010; Douglas & Albro, 2014)? How can instructional practices be intensified to facilitate greater comprehension of grade-level texts (Vaughn, Zumeta, Wanzek, Cook, & Klingner, 2014)?

Theoretical Framework

Theories and models of comprehension suggest that in order to read for understanding, readers must construct a coherent mental representation of the situation described by the text, called the situation model (Kintsch, 1988; van den Broek, Risden, Fletcher, & Thurlow, 1996). These situation-based representations are influenced by lower- and higher-level cognitive processes (van den Broek, Beker, & Oudega, 2015). With respect to lower-level cognitive processes, theories and models of comprehension suggest that comprehension depends on the reader's ability to quickly retrieve semantic information at the word level, which is influenced by the reader's decoding, fluency, and vocabulary knowledge (Gough & Tunmer, 1986; LaBerge & Samuels, 1974; Perfetti, 2007). With respect to higher-level cognitive processes, research has consistently demonstrated the importance of making inferences (Ahmed et al., 2014; Cromley & Azevedo, 2007; Kintsch, 1988; Oakhill & Cain, 2012; van den Broek, 1990), the ability to strategically monitor one's understanding of text (Cromley & Azevedo, 2007; Oakhill, Hartt, & Samols, 2005; van den Broek et al., 1996), general knowledge (Cromley & Azevedo, 2007; Cromley, Snyder-Hogan, & Luciw-Dubas, 2010), and working-memory capacity (Arrington, Kulesz, Francis, Fletcher, & Barnes, 2014; Cowan, 2010).

Importance of Inference Making for Comprehension

A major advance in reading-comprehension research concerns the central role of inference making during reading. Inference making is the process of integrating information within text and between the text and one's general knowledge of the topic (O'Brien, Cook, & Lorch, 2015). Skilled readers form two general types of inferences that are consistently, if not automatically, made during reading (Graesser, Li, & Feng, 2015; McKoon & Ratcliff, 1992). One type of inference—text-based inferences—links current information in text to information that was previously read. Knowledge-based inferences integrate current information in text with one's prior knowledge of the topic. Text- and knowledge-based inferences help the reader to fill in gaps in the text. In addition, inferences help the reader to build both sentence-level meaning that connects important information between adjacent sentences (i.e., local coherence) and discourse-level meaning that connects information across remote sections of text (i.e., global coherence; Kendeou, 2015; McNamara & Magliano, 2009).

Converging research has demonstrated that inference making represents the cornerstone of comprehension among elementary-grade readers (Cain & Oakhill, 1999; Oakhill & Cain, 2012; Oakhill, Cain, & Bryant, 2003) as well as skilled adult readers (Cromley et al., 2010; Cromley & Azevedo, 2007; Kendeou, 2015). In addition, child-based research provides direct evidence that inference making is not only a by-product of reading comprehension but also a possible mechanism that facilitates the development of reading comprehension among elementary-grade readers (Cain & Oakhill, 1999; Oakhill & Cain, 2012). Nevertheless, the importance of inference making among adolescents has only recently been demonstrated (Ahmed et al., 2014; Cromley & Azevedo, 2007).

Ahmed et al. (2014) examined the validity of the direct and inferential mediation model of reading comprehension (Cromley & Azevedo, 2007) among 1,196 students in Grades 7–12. Results indicated that when shared method variance was controlled for, inference making had the largest direct effect on reading comprehension after controlling for word reading, vocabulary knowledge, general knowledge, and strategy use; and that inference making also mediated some of the direct effect of knowledge on reading comprehension. This study adds to the growing body of research suggesting that inference making is one of the unique and significant predictors of comprehension (Ahmed et al., 2014; Cain, Oakhill, Barnes, & Bryant, 2001; Cromley & Azevedo, 2007; Hannon & Daneman, 1998; Kendeou, Bohn-Gettler, White, & van den Broek, 2008; Oakhill & Cain, 2012).

Inference Making for Adolescent Readers

The importance of inference making among adolescent readers, in particular, is further supported by recent research revealing individual differences in text- and knowledge-based inferences among students in middle and high school. For example, Barth, Barnes, Francis, Vaughn, and York (2015) reported that text-based inferences continue to develop in both rate and accuracy across the secondary grades. However, relative to skilled comprehenders, less skilled comprehenders are less accurate and slower at forming text-based inferences that link information that is close in proximity and necessary to maintain local coherence. Less skilled comprehenders also have difficulty making inferences that link information that is separated in text and required to maintain global coherence. In addition, relative to skilled comprehenders, less skilled adolescent comprehenders have difficulty in knowledge-to-text integration processes that facilitate sentence-level processing when reading (Barnes, Ahmed, Barth, & Francis, 2015). The importance of this recent research is that it challenges what has been assumed about reading comprehension among students in the secondary grades, namely that inference making would not demonstrate much developmental change after later elementary school and that the difficulties of adolescent readers lie in integrating information across large, not small, textual distances (Barth et al., 2015).

The Impact of Explicitly Teaching Inference Making

Given that inference making is the strongest predictor of comprehension among adolescent readers (Ahmed et al., 2014), and that less skilled adolescent readers are less accurate and slower at forming text- and knowledge-based inferences relative to skilled readers (Barnes et al., 2015; Barth et al., 2015), explicit instruction in inference making may lead to significant improvements in reading comprehension among struggling readers in the middle grades. This notion is supported by a recent synthesis (Hall, 2016) and meta-analysis (Elleman et al., 2011) examining the effects of teaching inference making on reading comprehension among students in the elementary and secondary grades.

Hall (2016) conducted a synthesis to assess the effectiveness of inference instruction at improving reading outcomes among less skilled comprehenders, as well as the features of interventions associated with significant gains in inference making and reading comprehension. A total of nine studies were synthesized (one single-case-design and eight group-design studies) across Grades 2–9. Mean effect sizes for group-design studies ranged from g = 0.72 to g = 1.85 for experimental measures of inference making; and from g = −0.03 to g = 1.96 for standardized measures of reading comprehension following, on average, 12.6 hr of instruction (range = 2.67 to 26.67). Features of instruction associated with significant improvements in reading comprehension included engaging students in the activation of prior knowledge to facilitate text-to-self connections (i.e., knowledge-based inferences); and identifying key or clue words in order to integrate important ideas in text (i.e., text-based inferences) or fill in textual gaps by integrating key words with relevant prior knowledge (i.e., knowledge-based inferences).

Elleman et al. (2011) conducted a meta-analysis of 24 inference intervention studies conducted across Grades K–12. Results indicate that explicit instruction in inference making is effective for both skilled (d = 0.55) and less skilled (d = 0.80) comprehenders, with less skilled comprehenders responding more positively to explicit inference instruction than their more skilled peers following less than 10 hr of instruction. Among less skilled comprehenders, results also indicate not only improved accuracy in forming inferences but also significant gains in literal comprehension of text. Features of instruction associated with significant improvements in reading comprehension included practice answering inference questions during or after reading (e.g., text- or knowledge-based inferences); identification of key or clue words in order to integrate important information in text (i.e., text-based inferences) or integrate information in text with relevant knowledge of the topic (i.e., knowledge-based inferences); and connecting ideas in text with relevant knowledge through explanation and elaboration (i.e., knowledge-based inferences).

Importance of Knowledge in Inference Making

Across both reviews, studies in which students were taught how to integrate prior knowledge with new information in text were effective at increasing comprehension, especially for less skilled comprehenders. This is important, given that many less skilled comprehenders exhibit knowledge deficits in comparison to skilled comprehenders (e.g., Chiesi, Spilich, & Voss, 1979; Langer, 1981) and that a frequent source of inference errors is the selection of relevant information from text or prior knowledge required to make the inference (Cain et al., 2001). Because less skilled comprehenders have difficulty identifying the relevant pieces of information that should be integrated to form inferences and frequently lack relevant background knowledge, this presents a catch-22. That is, students must make inferences to understand what they read and to acquire new knowledge, but they must also have prior knowledge to make appropriate inferences (McNamara, O'Reilly, & de Vega, 2007). Therefore, effective inference interventions would likely need to explicitly teach students how to identify relevant information from text required to form inferences while simultaneously building the necessary knowledge so that they can also practice how to integrate relevant prior knowledge with information from text.

Inference Studies Conducted Among Students in the Secondary Grades

To date, only two inference intervention studies have been conducted among students in the secondary grades. Fritschmann, Deshler, and Schumaker (2007) examined the effectiveness of teaching eight ninth-grade students with learning disabilities a five-step inference strategy for answering a variety of inference questions (i.e., main idea/summarizing, purpose, predicting, and clarification questions) frequently found on standardized assessments of reading comprehension. Using a multiple-baseline across-subjects design, results indicate that following approximately 15 hr of instruction, students made significant gains on criterion-based and standardized measures of reading comprehension. In more recent work, Elbro and Buch-Iversen (2013) taught general-education students in sixth grade (n = 236) how to identify relevant information in text and retrieve general knowledge required to form knowledge-based inferences using graphic organizers. Following eight 30-min sessions, a large training effect was found on a researcher-developed measure of inference making as well as a standardized assessment of reading comprehension.

In sum, the results of these two inference studies conducted among students in the secondary grades align with the recent syntheses by Hall (2016) and Elleman et al. (2011) that suggest that inference making is malleable to instruction, with improvements in inference making associated with significant gains in reading comprehension. However, there is an overall limited amount of research that has been conducted among struggling readers in the middle grades. Further, several methodological weaknesses make it difficult to arrive at generalizations and form recommendations regarding best practices for teaching inference making to older struggling readers. These methodological weaknesses include lack of random assignment of students to condition (Elbro & Buch-Iversen, 2013; Fritschmann et al., 2007), single observation or direct assessment of fidelity of implementation (Elbro & Buch-Iversen, 2013), and failure to teach both knowledge- and text-based inferences—the two most frequent types of inferences generated by readers (Elbro & Buch-Iversen, 2013; Fritschmann et al., 2007). For these reasons, there appears to be a need for additional intervention research on the effects of explicitly teaching inference making and content knowledge to struggling readers in the middle grades.

Study Purpose

Given the weakness of past research and our recognition that problems with inference making are likely multifaceted, the purpose of the current study was to examine the effectiveness of a multistrategy inference intervention designed to teach text- and knowledge-based inferences among middle-grade struggling readers. To accomplish this, we designed a multistrategy intervention on the basis of inference strategies that have been shown to be consistently effective for less skilled comprehenders (Elleman, 2011; Hall, 2016). Taking into consideration the likely knowledge deficits of less skilled comprehenders, we also carefully selected and sequenced narrative and informational texts so that students would build a knowledge base about ancient Egypt. By using text and discussions about text to build content knowledge, students would have the requisite knowledge with which to practice the inference-making strategies taught in the inference intervention program. Therefore, our first goal was to determine whether the students acquired the requisite knowledge base from the texts they read. We hypothesized that students in the treatment group would build sufficient content knowledge through reading text and participating in text-based discussions. The second goal was to determine whether this multistrategy inference intervention would lead to improvements in inference making as measured by a researcher-designed and a criterion-referenced measure of inference making. We hypothesized that explicit practice in strategies targeted to improve students' ability to form an accurate situation model would lead to significant improvements in inference making because the instruction, practice, and tutor feedback would help readers identify and integrate relevant information in text with their prior knowledge. Our third goal was to determine whether this multistrategy inference intervention would lead to significant improvements in reading comprehension as measured by a general, standardized measure of reading comprehension. We hypothesized that gains in inference making would lead to significant improvements in reading comprehension, given that previous syntheses report significant gains in reading comprehension following inference interventions of relatively short duration (Elleman, 2011; Hall, 2016) and that previous modeling of the direct and inferential mediation model reports a significant direct effect of inference making on reading comprehension (Ahmed et al., 2014; Cromley et al., 2010; Cromley & Azevedo, 2007).

Method

Participants

Study participants (n = 490) were students from one middle school serving Grades 6–8 in a midsize city in the Midwestern United States. The middle school served a student population that was 2.7% Asian American, 25.3% Black, 9.6% Hispanic, 0.4% Indian American, 6.7% multirace, and 55.3% White. Fifty-eight percent of students were economically disadvantaged as measured by participation in the school's free- and reduced-lunch program. Participation of the middle school was based on the willingness of the principal to cooperate and facilitate data collection and intervention delivery on the campus.

Potential participants were identified through a three-step process. First, on the basis of the previous year's scores on the state-mandated reading-comprehension competency test, the Missouri Assessment Program (Missouri Department of Elementary and Secondary Education, 2013), students who performed at or below “Basic” were identified as struggling readers. Second, the students were administered the STAR Reading Assessment (Renaissance Learning, 2014). Students who performed below the 25th percentile were then selected to participate in a supplemental reading class offered daily by special-education teachers (n = 80). Students were excluded from participation in the supplemental reading class if the school identified them as Limited English Proficient or as having a significant disability such as blindness, severe cognitive impairment, or severe behavioral disability. All students participating in the supplemental reading class who consented to participate in the study (n = 74) were administered the Wechsler Individual Achievement Test–Third Edition (WIAT-III; Wechsler, 2009) Reading Comprehension subtest. Students performing below the 25th percentile on the WIAT-III Reading Comprehension subtest were randomized within grade to treatment (n = 33) and comparison (n = 33) conditions. Eight students were excluded from further participation in the study because they performed at or above the 25th percentile.

The final sample consisted of 66 students between the ages of 12 and 15 years (M = 13.7, SD = 0.94). Most were White (48.5%); the rest reported their ethnicities as Black (36.4%), Hispanic (7.6%), multirace (1.5%), and other (1.5%). Thirty percent were served by special education and 53% of the sample were boys (see Table 1). Students randomly assigned to treatment did not differ from those assigned to the control condition on demographic variables such as race, χ²(4, N = 66) = 2.86, p =.56; gender, χ²(1, N = 66) = 2.98, p = .08; special-education status, χ²(1, N = 66) = 0, p = 1.0; or grade level, χ²(1, N = 66) = 2.98, p = .08. Students assigned to treatment did not significantly differ from those assigned to the control condition on measures administered at pretest—the Kaufman Brief Intelligence Test–Second Edition (KBIT-2; Kaufman & Kaufman, 2004) Matrices subtest, which was administered at posttest to assist in describing the sample, and the Missouri Assessment Program Reading assessment, which was administered in the spring of the previous school year—Test of Word Reading Efficiency (TOWRE; Torgesen, J., Wagner, R., & Rashotte, 2012) Sight Word Efficiency, t(64) = −1.18, p = .24; TOWRE Phonemic Decoding Efficiency, t(63) = −.57, p = .57; KBIT-2 Verbal Knowledge, t(48) = .05, p = .96; KBIT-2 Matrices, t(63) = −.07, p = .95; WIAT-III Reading Comprehension, t(64) = .06, p = .95; Egyptian Content Knowledge Assessment, t(64) = −.17, p = .87; Qualitative Reading Inventory–5 (QRI-5; Leslie & Caldwell, 2011) Retell, t(57) = .67, p = .51; and Missouri Assessment Program Reading level, χ²(2, N = 59) = 1.62, p = .45.

Table 1.

Participant characteristics.

Characteristic	Treatment group (n = 33)	Comparison group (n = 33)
Race
White	46%	52%
Black	42%	30%
Hispanic	9%	6%
Other	3%	0%
Multiple	0%	3%
Gender
Male	18%	32%
Female	82%	68%
Special education	30%	30%
Missouri Assessment Program Reading score (M [SD])	630.06 (22.17)	635.48 (28.63)
Age (years; M [SD])	13.72 (0.99)	13.62 (0.90)
Grade
6	8	8
7	12	13
8	13	12

Following pretesting, five students (one in the treatment group and four in the comparison group) were dropped from the study because they were not present at posttesting. This represents approximately 8% of study participants. The remainder of this article focuses on the 32 students who actually received the intervention and 30 students in the comparison group, who received remedial reading instruction provided by the school.

Intervention Instructors

Six tutors, hired and trained by the research team, provided intervention to study participants. Tutors were undergraduates in the program on communication sciences and disorders. The investigators provided the interventionists with 10 hr of professional development prior to tutoring as well as an additional 5 hr of professional development related to the intervention over the course of the 10-day intervention trial. Professional development included training on the research supporting the intervention design, features of effective comprehension instruction, inference-strategy instruction, behavior-management strategies for middle-grade struggling readers, delivery of effective feedback, and general information about middle-grade struggling readers. The majority of the professional development focused on familiarizing the tutors with the scripted and semiscripted lessons. Each strategy was modeled for the tutors; tutors then practiced introducing each strategy with feedback. After tutors practiced the fully scripted introduction lessons, the semiscripted portion of the practice lessons were modeled for them. Tutors learned how to use the provided questioning suggestions to prompt student responses, as well as the feedback suggestions for student responses (see Appendix).

Description of treatment- and Comparison-Group Activities

Treatment Group

Students in the treatment condition met with one of six trained research interventionists, in groups of two or three students per tutor, for 45 min/day for a total of 10 intervention days and 7.5 hr of instruction. Intervention was delivered 4 days/week.

The multistrategy inference intervention was designed to simultaneously build content knowledge and teach students how to generate text- and knowledge-based inferences. Students were first taught to think about reading like a detective solves a crime. The strategies were taught within this reading-detectives theme. Over the first 2 days, each inference strategy was introduced explicitly using fully scripted lessons and researcher-designed passages. The tutors modeled each strategy, and then the students practiced the strategies with tutor feedback. In the third through 10th lessons, students learned to apply the strategies while reading authentic narrative and expository texts. They were also taught to identify and answer inferential questions. Expository text included short passages from Egyptworld (Caldwell, 2013), with a Lexile level of 1,160, and Mummies and Pyramids (Osborne & Osborne, 2001), a 106-page, high-interest text about ancient Egypt with a Lexile level of 650. Students read the narrative text Tut, Tut (Scieszka, 2006), a 73-page book with a Lexile level of 700. For these lessons, previous material was reviewed using fully scripted lessons, and the activities that occurred just before and during reading (i.e., text clues for clarification of vocabulary, activation and integration of prior knowledge, understanding of author purpose and character perspective) were taught using semiscripted lessons. Although these portions of the lesson were partially scripted, tutors were provided with all of the connections within and between texts so they could prompt students to identify relevant connections and gain a deeper understanding of the material. Tutors were also provided with question stems and suggestions for feedback in each lesson. Feedback for the last activity, identifying and answering inferential questions, was fully scripted for the tutors. Over time, the level of support given to students by the tutors was gradually reduced to encourage independent use of the strategies.

Using text clues for clarification. Target words and phrases were selected on the basis of the importance of the vocabulary and ideas to the comprehension of the text. Students learned to use text clues to clarify unknown target words or ideas in the text. They were taught to periodically ask themselves if the text made sense; if not, they were to reread and identify clues that might clarify the text. If that failed, students were instructed to ask the group for help. To give practice at clarifying and to add novelty, each day the students would find a word covered with a sticky note. When encountering the covered word or phrase, they would use the text-clue strategy to predict the word and would then reveal the word to check their prediction.

Activating and integrating prior knowledge. Students were taught to make text connections within the text, between texts, and between the texts and their prior knowledge. Students were required to justify how their connections helped in understanding the text or answering a question about the text. Struggling readers have been shown to make irrelevant connections when generating inferences (e.g., Williams, 1993). Therefore, when a student made an irrelevant connection to his or her background knowledge, the tutor provided feedback asking the student to evaluate whether the particular connection would help to better understand the text. This helped the students to inhibit irrelevant background knowledge that otherwise might have distracted them from focusing on gaining meaning from the text.

Understanding character perspectives and author's purpose. In the narrative texts, students were taught to make inferences about a character's motive and intent using text clues. They were encouraged to determine why characters acted in certain ways and to identify when characters were acting oddly on the basis of the information given by the author. In narrative and expository text, students were taught to note repeated or lengthy information and justify why this information was important to the text.

Answering inference questions. Students were taught to identify whether questions required making an in-text inference or an elaborative inference using information from the text and background knowledge. Students were also taught to identify vocabulary questions and main-idea questions that required synthesis. Students were taught to find the most important information and explain why that information was the most important. After identifying each question type, students used the appropriate strategy to answer the item. More detailed information about the scope and sequence of the lessons, as well as lesson examples, can be found in the Appendix.

Comparison Group

Students randomized to the control condition continued to participate in their supplemental, remedial-reading intervention class for the duration of the 10-day intervention trial. For Days 1 through 5, they took turns reading aloud Pygmalion, a play by George Bernard Shaw (1916), and answering comprehension questions. On Days 6 through 9, they practiced assigned parts of the play. On the 10th day, students performed the play for parents and various teachers.

Fidelity of Implementation

Implementation fidelity checklists were completed by each tutor for every intervention session. In addition, they were completed by the primary investigator for five of 10 sessions (50%) per tutor. The checklist documented completion of the four intervention components. Using percent agreement, implementation validity was calculated as 97%, demonstrating that the intervention was implemented with high fidelity.

Procedure

Given that lower-level (i.e., word reading and verbal knowledge) and higher-level (i.e., inference making and knowledge) skills are associated with reading comprehension among adolescent readers (Ahmed et al., 2014; Barth et al., 2015), students were assessed with a pretest battery that assessed word-reading efficiency, verbal knowledge, reading comprehension, content knowledge, and inference making. At posttest, students were assessed with a battery that assessed reading comprehension, content knowledge, and inference making. They were assessed both individually and in a group format. At pretest, word-reading efficiency, verbal knowledge, and reading comprehension were assessed individually; content knowledge and inference making were assessed in a group format. At posttest, which was administered the day after the intervention concluded, reading comprehension was assessed individually; content knowledge and inference making were assessed in group format.

Students were tested by tutors who completed an extensive training program on test administration and scoring. All students were tested in a quiet classroom designated by the principal, during a time identified by the principal. Following data collection, all student test packets were checked for accuracy of scoring and administration. Packets were first scored by the tutor who tested the student; they were then rescored by two additional tutors who did not directly test the student. Interscorer agreement exceeded 90 percent accuracy for all assessments in the pretest and posttest batteries.

Measures

Measures to Define the Sample

TOWRE-2. The Sight Word Efficiency subtest is a measure of an individual's ability to pronounce phonemically regular words accurately and fluently. There are 104 items in this subtest, and the total number of words read accurately in 45 s is recorded. The Phonemic Decoding Efficiency subtest is a measure of an individual's ability to pronounce phonemically regular nonwords accurately and fluently. There are 76 items in this subtest, and the total number of nonwords read correctly within 45 s is recorded. The composite standard score for the two subtests was used in analyses. Among students in Grades 6–8, alternate forms and test–retest reliability coefficients were at or above .90. This test was administered to provide a fuller picture of the student sample's word recognition skills.

KBIT-2. The Matrices subtest is an individually administered assessment designed to measure fluid thinking or the ability to solve new problems by perceiving relationships and completing analogies. It consists of 46 multiple-choice nonverbal items that involve visual stimuli, both meaningful (people and objects) and abstract (designs and symbols). For each item, students either point to the correct response or say its letter. The Verbal Knowledge subtest is an individually administered assessment designed to measure receptive vocabulary and general information (e.g., nature, geography). The student is required to choose one of six illustrations that best corresponds to a tutor question. The KBIT-2 also has a Riddles subtest that was not used; instead, the Verbal Knowledge score was prorated for the verbal domain, and therefore verbal and nonverbal standard scores were used. Internal-consistency coefficients (split half) for the nonverbal scores for ages 11 through 18 years (covering Grades 6 through 12) range from .86 to .91 (M = .88). This measure was administered to provide a fuller picture of the general cognitive skills of the student sample.

Outcome Measures

WIAT-III. The WIAT-III Reading Comprehension subtest is an individually administered standardized assessment of reading comprehension tapping comprehension skills taught in school or used in daily life. Students read different types of passages and respond to questions involving understanding of the content. Total standard score is used in our analyses. Internal consistency exceeds .90 for students in Grades 6–8.

Egyptian Content Knowledge Assessment (Elleman, Barth, & Oslund, 2015). The Egyptian Content Knowledge Assessment is a 25-item custom assessment measuring basic knowledge of ancient Egypt. Of the 25 items, 11 are multiple choice and 14 are constructed response. The total raw score was used in our analyses. Internal consistency among students in Grade 5, calculated using Cronbach's alpha, was .89 (Elleman et al., 2015); it was .85 among students in this sample.

Criterion-referenced measure of recall and inference making. The QRI-5 is an informal reading inventory designed to assess the quality of the readers' unaided recall and their understanding of text when prompted with questions. For the purposes of this study, students completed the QRI-5 as a group using two sixth-grade expository passages on ancient Egypt (i.e., “Nile” and “Building Pyramids”). The Nile passage was 294 words in length, with a Lexile rating of 850, and included six literal and four inferential comprehension questions. The Building Pyramids passage was 304 words in length, also with a Lexile rating of 850, and included seven literal and three inferential comprehension questions. Students were asked to read each passage, write a retell of the passage, then write their answers to the comprehension questions. Two scores were generated: QRI-5 Recall raw score and QRI-5 Inference-making raw score. Internal consistency was .83.

Analysis Plan

Analyses were conducted in the context of the generalized linear model (Green & Silverman, 1994; Littell, Stroup, & Freund, 2002) using SAS. Analysis-of-covariance models were fitted with pretest performance as the primary covariate, comparing treatment versus comparison groups on the Egyptian Content Knowledge Assessment, QRI-5 Recall, QRI-5 Inference-making, and WIAT-III Reading Comprehension. Linear relations between the outcomes and covariates were confirmed through visual inspection, and Levene's test for equality of variance confirmed homogeneous variances of outcomes between groups. Assumptions about homoscedasticity and normality of residuals were examined. In a final step, the Benjamini–Hochberg method (Benjamini & Hochberg, 1995) was applied separately for outcome measures to correct the critical p value and protect against type I error due to multiple testing.

Results

Preliminary Analyses

Table 2 presents the means and standard deviations for each of the measures administered at pretest as well as the KBIT-2 Matrices, which was administered at posttest. These descriptive statistics are provided separately for the treatment and comparison groups.

Table 2.

Means and standard deviations by condition.

Variable	Treatment group						Comparison group
	Pretest			Posttest			Pretest			Posttest
	n	M	SD	n	M	SD	n	M	SD	n	M	SD
WIAT-III-SS	32	82.84	5.44	32	88.91	6.38	29	82.66	6.25	29	85.03	6.33
ECKA	32	7.59	3.03	32	13.00	3.89	29	7.38	2.76	29	8.21	3.59
QRI-5-Comp	32	11.50	3.31	32	12.79	3.44	29	11.72	4.72	29	10.69	4.56
QRI-5-Retell	32	11.63	6.75	32	10.94	5.69	29	13.31	6.50	29	11.28	8.63
KBIT-2-VK	32	85.72	6.48				29	85.04	8.44
KBIT-2-Matrices				32	92.66	15.32				29	89.28	13.82
TOWRE-PDE	32	85.72	12.31				29	82.96	10.58
TOWRE-SWE	32	89.38	10.39				29	86.45	8.72

Note. WIAT-III-SS = Wechsler Individual Achievement Test–Third Edition Reading Comprehension standard score. ECKA = Egyptian Content Knowledge Assessment raw score. QRI-5-Comp = Qualitative Reading Inventory–5 Comprehension total raw score. QRI-5-Retell = Qualitative Reading Inventory–5 Retell total raw score. KBIT-2-VK = Kaufman Brief Intelligence Test–Second Edition Verbal Knowledge standard score. KBIT-2-Matrices = Kaufman Brief Intelligence Test–Second Edition Matrices standard score. TOWRE-PDE = Test of Word Reading Efficiency Phonemic Decoding Efficiency standard score. TOWRE-SWE = Test of Word Reading Efficiency Sight Word Efficiency standard score.

Differences Between Treatment and Comparison Groups

Four separate analyses of covariance were performed to evaluate mean differences between the treatment and comparison groups. In each instance, the pretest score was used as the covariate. Pretest adjusted and unadjusted means and standard deviations, with the results from the analyses of covariance, are presented in Table 3. In addition, the table provides standardized effect sizes calculated for treatment effects using pretest adjusted measures and observed standard deviations. We found significant treatment effects on WIAT-III Reading Comprehension, F(1, 58) = 6.88, p = .0111, g = 0.46, and QRI-5 Inference-making, F(1, 58) = 4.68, p = .035, g = 0.36. In terms of content-knowledge acquisition, significant difference was found on the Egyptian Content Knowledge Assessment, F(1, 58) = 20.31, p < .0001, g = 1.37. And last, significant differences were not found on QRI-5 Retell, F(1, 58) = 0.10, p = .7543, g = −0.01. We then applied the Benjamini–Hochberg method separately for each assessment to correct the critical p value and protect against type I error due to multiple testing. WIAT-III Passage Comprehension and the Egyptian Content Knowledge Assessment remained significant; QRI-5 Inference-making failed to remain significant.

Table 3.

Effects of intervention on comprehension and knowledge.

Assessment	Variable	Treatment group	Comparison group
WIAT-III-SS	n	32	29
	Adjusted M	88.86	85.09
	SD	0.99	1.04
	Hedge's g (SE)	0.46
	F(1, 58)	6.88
	p	.0111
ECKA	n	32	29
	Adjusted M	12.95	8.21
	SD	0.63	0.61
	Hedges's g (SE)	1.37
	F(1, 58)	26.73
	p	< .0001
QRI-5-Comp	n	32	29
	Adjusted M	12.73	10.65
	SD	0.66	0.70
	Hedges's g (SE)	0.36
	F(1, 58)	4.68
	p	.0346
QRI-5-Retell	n	32	29
	Adjusted M	11.35	10.82
	SD	1.14	1.20
	Hedges's g (SE)	−0.01
	F(3, 62)	0.10
	p	.7543

Note. WIAT-III-SS = Wechsler Individual Achievement Test–Third Edition Reading Comprehension standard score. ECKA = Egyptian Content Knowledge Assessment raw score. QRI-5-Comp = Qualitative Reading Inventory–5 Comprehension total raw score. QRI-5-Retell = Qualitative Reading Inventory–5 Retell total raw score.

Discussion

This randomized controlled trial examined the effectiveness of a multistrategy inference intervention that emphasized the use of text clues to clarify the meaning of novel words, the activation and use of prior knowledge, understanding of character perspectives, and practice answering inference questions. The content of the intervention was selected to build the necessary knowledge required to form knowledge-based inferences and to facilitate the integration of important information in the text (text-based inferences). The intervention goals were chosen to reflect our interest in understanding whether improvements in inference making (after controlling for knowledge) would lead to improved reading comprehension among struggling readers in the middle grades. We found positive effects of the intervention on knowledge learned in the intervention as well as on a standardized measure of reading comprehension.

Our first goal of this study was to determine whether students acquired the requisite knowledge base with which to form knowledge-based inferences and potentially facilitate greater accuracy in integrating relevant information in the text (text-based inferences). We hypothesized that the intervention would allow students to build sufficient content knowledge through reading text and participating in text-based discussions. Narrative and informational texts were carefully selected and sequenced to build content knowledge to facilitate the use of the inference strategies and formation of accurate inferences. Results revealed significant effects on a proximal measure of Egyptian-content knowledge, suggesting that text-based discussions that provided students repeated practice in integrating important information in text with relevant background knowledge led to significant improvements in students' recall of content-specific knowledge.

The second goal was to determine whether this multistrategy inference intervention would lead to improvements in inference making as measured by a researcher-designed (i.e., Egyptian Content Knowledge Assessment) and a criterion-referenced (i.e., QRI-5 Inference-making) measure of inference making. We hypothesized that explicit practice in strategies designed to help readers identify relevant information and integrate that relevant information with their knowledge of the topic would lead to significant improvements in inference making. Although practical gains were made on the QRI-5 Inference-making measure by students in the intervention, these significant effects were not maintained after we controlled for multiple statistical comparisons. In addition, we did not find significant gains on the closely aligned measure of inference making (i.e., Egyptian Content Knowledge Assessment).

Why did students in the treatment group not make significant gains on measures of inference making that were aligned with the content of the intervention (i.e., ancient Egypt)? It is unclear why these proximal measures were not sensitive to changes in inference making. Although both measures comprised passages on ancient Egypt, the passages covered information that did not overlap with content taught in the intervention. It is likely that the content of these texts was too distantly related to the information that students were learning in the intervention. As a consequence, the students were unable to efficiently and accurately make inferences that required the integration of information in text and of information in text with general knowledge of the topic. In other words, knowledge of ancient Egypt was improved and likely influenced the use of comprehension monitoring and the mechanisms that initiate inference making for the thematically related texts used across the 10 days of intervention, but the knowledge base was not developed sufficiently to facilitate these higher-level processes for similar but more distantly related texts that comprised the proximal measures of inference making.

Our final goal was to determine whether this multistrategy inference intervention would lead to significant improvements in reading comprehension as measured by a general, standardized measure of reading comprehension. We hypothesized that gains in inference making would lead to significant improvements in reading comprehension, given that previous syntheses report significant gains in reading comprehension following inference interventions of relatively short duration (Elleman, 2011; Hall, 2016), and that previous modeling of the direct and inferential mediation model reports a significant direct effect of inference making on reading comprehension (Ahmed et al., 2014; Cromley & Azevedo, 2007; Cromley et al., 2010). Results revealed a significant effect on the WIAT-III Reading Comprehension subtest, a standardized assessment of reading comprehension, following only 10 days of instruction.

These findings are important in light of recent randomized control trials, examining the effectiveness of multicomponent reading interventions designed to remediate the reading difficulties of middle-grade struggling readers, that report effects on standardized measures of comprehension ranging from small to negligible (Vaughn et al., 2010, 2011, 2012), with more significant effects requiring more than 1 year of intervention (Roberts, Vaughn, Fletcher, Stuebing, & Barth, 2013). This suggests either that current intervention approaches have not been intensive enough to budge the intractable deficits present for middle-grade struggling readers (Vaughn et al., 2013) or that the components do not adequately build the types of higher-order reading skills involved in comprehension itself. Because this study demonstrated significant effects on a standardized measure of reading comprehension following only 10 days of intervention, incorporation of explicit practice in forming text- and knowledge-based inferences while simultaneously building content knowledge may improve the effectiveness of current reading-comprehension interventions for older struggling readers; however, future research is required to substantiate this claim.

Clinical Significance

This study is an example of an early efficacy study. We were interested in integrating intervention components previously reported to be effective in developing the inference-making skills of typically developing readers and struggling readers in the elementary and secondary grades (Elleman et al., 2011; Hall, 2016). Because we were interested in assessing the potential benefit of emphasizing the use of text clues for clarification of unknown words, the activation and use of prior knowledge, the understanding of character perspectives, and the answering of inference questions, we included proximal measures closely aligned to the intervention and a standardized measure of reading comprehension. Of importance is that significant effects of intervention were found for the proximal measure of content knowledge and a standardized measure of reading comprehension following only 10 days of intervention, delivered daily in small groups of three students. This is clinically important, given that the majority of intervention research conducted among middle-grade struggling readers reports small to negligible effects on standardized measures of reading comprehension following 1 year of intensive intervention, delivered daily in small groups of students by highly trained teachers (Vaughn et al., 2010, 2011, 2012).

Study Limitations and Future Research

This study provides preliminary information about the effectiveness of a multicomponent inference intervention. Although our findings support ongoing inquiry in this area, the preliminary nature of the study revealed a couple of important limitations to consider for future research. First, improvements in inference making were not found following intervention. Future research should further develop and refine measures of inferential comprehension, which is a critical barrier to advancing our understanding of how inference making develops and what the effects are of inference interventions among adolescent struggling readers. Second, the intervention was of short duration and included only one standardized measure of reading comprehension that was administered individually to students. Future research should include standardized measures of listening comprehension and reading comprehension to understand the extent to which intervention response transfers and generalizes across various comprehension constructs and assessment formats. Third, the sample size for this study was small. Although students were randomized to condition within grade, future research should increase the sample size and diversify the demographics of the sample in order to understand how inference interventions generalize across the various subgroups of middle-grade struggling readers. And last, the intervention used various strategies for increasing students' knowledge building and accuracy in forming inferences. In consequence, one, several, or all components could have contributed to the significant improvements in knowledge acquisition and comprehension. Therefore, future research should isolate the effects of teaching knowledge building and inference making in order to understand how these mechanisms function and lead to improved reading comprehension.

Appendix

Reading-Detectives Scope and Sequence

Instructional activities	Texts
Introduction to being a reading detective; introduction to using clues to clarify unknown words and ideas	Short researcher-designed passages
Introduction to activating and using prior knowledge; introduction to author's purpose and character motives	Short researcher-designed passages
Practice applying strategies during text reading with modeling and feedback; introduction to question types and answering questions after reading	Mummies and Pyramids
Practice applying strategies during text reading with modeling and feedback; review of question types and answering questions after reading	Mummies and Pyramids
Independent application of strategies on short warm-up passages; continued practice applying strategies during reading with feedback; practice answering inferential questions after reading	Egyptworld
	Tut, Tut

Funding Statement

This research was supported by Eunice Kennedy Shriver National Institute of Child Health and Human Development Grant K08 HD068545-01A1 (awarded to Amy E. Barth). The opinions expressed are those of the authors and do not represent views of the institute.