A Synthesis of Reading Interventions and Effects on Reading Comprehension Outcomes for Older Struggling Readers

Abstract

This article reports a synthesis of intervention studies conducted between 1994 and 2004 with older students (Grades 6–12) with reading difficulties. Interventions addressing decoding, fluency, vocabulary, and comprehension were included if they measured the effects on reading comprehension. Twenty-nine studies were located and synthesized. Thirteen studies met criteria for a meta-analysis, yielding an effect size (ES) of 0.89 for the weighted average of the difference in comprehension outcomes between treatment and comparison students. Word-level interventions were associated with ES = 0.34 in comprehension outcomes between treatment and comparison students. Implications for comprehension instruction for older struggling readers are described.

Although educators have historically emphasized improving students’ reading proficiency in the elementary school years, reading instruction for secondary students with reading difficulties has been less prevalent. As a result, secondary students with reading difficulties are infrequently provided reading instruction, thus widening the gap between their achievement and that of their grade-level peers. Recent legislation, such as the No Child Left Behind Act (NCLB; 2002), has prompted schools to improve reading instruction for all students, including those in middle and high school. Many secondary students continue to demonstrate difficulties with reading, and educators continue to seek information on best practices for instructing these students.

The National Assessment of Educational Progress (NAEP) administered a reading assessment in 2002 to approximately 343,000 students in Grades 4 and 8. According to the NAEP data, there was no significant change in progress for students between 1992 and 2002, and Grade 8 scores in 2003 actually decreased (Grigg, Daane, Jin, & Campbell, 2003). The NAEP also conducted a long-term trend assessment in reading, which documented performance from 1971 to 2004 for students ages 9, 13, and 17. Although scores for the 9-year-olds showed improvements compared to the scores for this age in 1971 and 1999, this was not the case for the 13- and 17-year-olds. Although the scores at the 75th and 90th percentile for the 13-year-olds significantly improved from 1971 to 2004, there were no significant differences between scores in 1999 and 2004. For the 17-year-olds, there were no significant differences at any of the percentiles selected in 2004, nor were there differences between the 1971 and 1999 scores. These data suggest that the education system is not effectively preparing some adolescents for reading success and that information on effective instructional practices is needed to improve these trends.

Expectations

Secondary students face increasing accountability measures along with a great deal of pressure to meet the demands of more difficult curricula and content (Swanson & Hoskyn, 2001). In the past decade, students have become responsible for learning more complex content at a rapid pace to meet state standards and to pass outcome assessments (Woodruff, Schumaker, & Deschler, 2002).

Our educational system expects that secondary students are able to decode fluently and comprehend material with challenging content (Alvermann, 2002). Some struggling secondary readers, however, lack sufficient advanced decoding, fluency, vocabulary, and comprehension skills to master the complex content (Kamil, 2003).

In a climate where many secondary students continue to struggle with reading and schools face increasingly difficult accountability demands, it is essential to identify the instruction that will benefit struggling secondary readers. Secondary teachers require knowledge of best practices to provide appropriate instruction, prevent students from falling farther behind, and help bring struggling readers closer to reading for knowledge and pleasure.

Comprehension Research

The ultimate goal of reading instruction at the secondary level is comprehension—gaining meaning from text. A number of factors contribute to students’ not being able to comprehend text. Comprehension can break down when students have problems with one or more of the following: (a) decoding words, including structural analysis; (b) reading text with adequate speed and accuracy (fluency); (c) understanding the meanings of words; (d) relating content to prior knowledge; (e) applying comprehension strategies; and (f) monitoring understanding (Carlisle & Rice, 2002; National Institute for Literacy, 2001; RAND Reading Study Group, 2002).

Because many secondary teachers assume that students who can read words accurately can also comprehend and learn from text simply by reading, they often neglect teaching students how to approach text to better understand the content. In addition, because of increasing accountability, many teachers emphasize the content while neglecting to instruct students on how to read for learning and understanding (Pressley, 2000; RAND Reading Study Group, 2002). Finally, the readability level of some text used in secondary classrooms may be too high for below-grade level readers, and the “unfriendliness” of some text can result in comprehension challenges for many students (Mastropieri, Scruggs, & Graetz, 2003).

The RAND Reading Study Group (2002) created a heuristic for conceptualizing reading comprehension. Fundamentally, comprehension occurs through an interaction among three critical elements: the reader, the text, and the activity. The capacity of the reader, the values ascribed to text and text availability, and reader’s activities are among the many variables that are influenced and determined by the sociocultural context that both shapes and is shaped by each of the three elements. This synthesis addresses several critical aspects of this proposed heuristic—the activity or intervention provided for students at risk and, when described in the study, the text that was used. Because the synthesis focuses on intervention research, questions about what elements of interventions were associated with reading comprehension were addressed. This synthesis was not designed to address other critical issues, including the values and background of readers and teachers and the context in which teachers and learners interacted. Many of the social and affective variables associated with improved motivation and interest in text for older readers and how these variables influenced outcomes are part of the heuristic of reading comprehension, but we were unable to address them in this synthesis.

Rationale and Research Question

Many of the instructional practices suggested for poor readers were derived from observing, questioning, and asking good and poor readers to “think aloud” while they read (Dole, Duffy, Roehler, & Pearson, 1991; Heilman, Blair, & Rupley, 1998; Jiménez, Garcia, & Pearson, 1995, 1996). These reports described good readers as coordinating a set of highly complex and well-developed skills and strategies before, during, and after reading so that they could understand and learn from text and remember what they read (Paris, Wasik, & Tumer, 1991). When compared with good readers, poor readers were considerably less strategic (Paris, Lipson, & Wixson, 1983). Good readers used the following skills and strategies: (a) reading words rapidly and accurately; (b) noting the structure and organization of text; (c) monitoring their understanding while reading; (d) using summaries; (e) making predictions, checking them as they read, and revising and evaluating them as needed; (g) integrating what they know about the topic with new learning; and (h) making inferences and using visualization (Jenkins, Heliotis, Stein, & Haynes, 1987; Kamil, 2003; Klingner, Vaughn, & Boardman, 2007; Mastropieri, Scruggs, Bakken, & Whedon, 1996; Pressley & Afflerbach, 1995; Swanson, 1999; Wong & Jones, 1982).

Previous syntheses have identified critical intervention elements for effective reading instruction for students with disabilities across grade levels (e.g., Gersten, Fuchs, Williams, & Baker, 2001; Mastropieri et al., 1996; Swanson, 1999). For example, we know that explicit strategy instruction yields strong effects for comprehension for students with learning difficulties and disabilities (Biancarosa & Snow, 2004; Gersten et al., 2001; National Reading Panel [NRP], 2000; RAND Reading Study Group, 2002; Swanson, 1999). We also know that effective comprehension instruction in the elementary grades teaches students to summarize, use graphic organizers, generate and answer questions, and monitor their comprehension (Mastropieri et al., 1996; Kamil, 2004).

However, despite improved knowledge about effective reading comprehension broadly, much less is known regarding effective interventions and reading instruction for students with reading difficulties in the middle and high school grades (Curtis & Longo, 1999). The syntheses previously discussed focused on students identified for special education, examined specific components of reading, and did not present findings for older readers. In recognition of this void in the research, the report on comprehension from the RAND Reading Study Group (2002) cited the need for additional knowledge on how best to organize instruction for low-achieving students. We have conducted the following synthesis to determine the outcome of comprehension, word study, vocabulary, and fluency interventions on reading comprehension of students in Grades 6 through 12. Furthermore, we extended the synthesis to include all struggling readers, not just those with identified learning disabilities. We addressed the following question: How does intervention research on decoding, fluency, vocabulary, and comprehension influence comprehension outcomes for older students (Grades 6 through 12) with reading difficulties or disabilities?

Method

For this synthesis, we conducted a comprehensive search of the literature through a three-step process. The methods described below were developed during prior syntheses conducted by team members (Kim, Vaughn, Wanzek & Wei, 2004; Wanzek, Vaughn, Wexler, Swanson, & Edmonds, 2006). We first conducted a computer search of ERIC and PsycINFO to locate studies published between 1994 and 2004. We selected the last decade of studies to reflect the most current research on this topic. Descriptors or root forms of those descriptors (reading difficult*, learning disab*, LD, mild handi*, mild disab* reading disab*, at-risk, high-risk, reading delay*, learning delay*, struggle reader, dyslex*, read*, comprehen*, vocabulary, fluen*, word, decod*, English Language Arts) were used in various combinations to capture the greatest possible number of articles. We also searched abstracts from prior syntheses and reviewed reference lists in seminal studies to assure that all studies were identified.

In addition, to assure coverage and because a cumulative review was not located in electronic databases or reference lists, a hand search of 11 major journals from 1998 through 2004 was conducted. Journals examined in this hand search included Annals of Dyslexia, Exceptional Children, Journal of Educational Psychology, Journal of Learning Disabilities, Journal of Special Education, Learning Disability Quarterly, Learning Disabilities Research and Practice, Reading Research Quarterly, Remedial and Special Education, and Scientific Studies of Reading.

Studies were selected if they met all of the following criteria:

• Participants were struggling readers. Struggling readers were defined as low achievers or students with unidentified reading difficulties, with dyslexia, and/or with reading, learning, or speech or language disabilities. Studies also were included if disaggregated data were provided for struggling readers regardless of the characteristics of other students in the study. Only disaggregated data on struggling readers were used in the synthesis.

• Participants were in Grades 6 through 12 (ages 11–21). This grade range was selected because it represents the most common grades describing secondary students. When a sample also included older or younger students and it could be determined that the sample mean age was within the targeted range, the study was accepted.

• Studies were accepted when research designs used treatment–comparison, single-group, or single-subject designs.

• Intervention consisted of any type of reading instruction, including word study, fluency, vocabulary, comprehension, or a combination of these.

• The language of instruction was English.

• At least one dependent measure assessed one or more aspects of reading.

• Data for calculating effect sizes were provided in treatment–comparison and single-group studies.

• Interrater agreement for article acceptance or rejection was calculated by dividing the number of agreements by the number of agreements plus disagreements and was computed as 95%.

Data Analysis

Coding procedures

We employed extensive coding procedures to organize pertinent information from each study. We adapted previously designed code sheets that were developed for past intervention syntheses (Kim, Vaughn, Wanzek, & Wei, 2004). The code sheet included elements specified in the What Works Clearinghouse Design and Implementation Assessment Device (Institute of Education Sciences, 2003), a document used to evaluate the quality of studies.

The code sheet was used to record relevant descriptive criteria as well as results from each study, including data regarding participants (e.g., number, sex, exceptionality type), study design (e.g., number of conditions, assignment to condition), specifications about conditions (e.g., intervention, comparison), clarity of causal inference, and reported findings. Participant information was coded using four forced-choice items (socioeconomic status, risk type, the use of criteria for classifying students with disabilities, and gender) and two open-ended items (age as described in text and risk type as described in text). Similarly, design information was gathered using a combination of forced-choice (e.g., research design, assignment method, fidelity of implementation) and open-ended items (selection criteria). Intervention and comparison information was coded using 10 open-ended items (e.g., site of intervention, role of person implementing intervention, duration of intervention) as well as a written description of the treatment and comparison conditions.

Information on clarity of causal inference was gathered using 11 items for true experimental designs (e.g., sample sizes, attrition, plausibility of intervention contaminants) and 15 items for quasiexperimental designs (e.g., equating procedures, attrition rates). Additional items allowed coders to describe the measures and indicate measurement contaminants. Finally, the precision of outcome for both effect size estimation and statistical reporting was coded using a series of 10 forced-choice yes–no questions, including information regarding assumptions of independence, normality, and equal variance. Effect sizes were calculated using information related to outcome measures, direction of effects, and reading outcome data for each intervention or comparison condition.

After extensive training (more than 10 hr) on the use and interpretation of items from the code sheet, interrater reliability was determined by having six raters independently code a single article. Responses from the six coders were used to calculate the percentage of agreement (i.e., agreements divided by agreements plus disagreements). An interrater reliability of .85 was achieved. Teams of three coded each article, compared results, and resolved any disagreements in coding, with final decisions reached by consensus. To assure even higher reliability than .85 on coding, any item that was not unambiguous to coders was discussed until a clear coding response could be determined. Finally, two raters who had achieved 100% reliability on items related to outcome precision and data calculated effect sizes for each study.

After the coding had been completed, the studies were summarized in a table format. Table 1 contains information on study design, sample, and intervention implementation (e.g., duration and implementation personnel). In Table 2, intervention descriptions and effect sizes for reading outcomes are organized by each study’s intervention type and design. Effect sizes and p values are provided when appropriate data were available.

TABLE 1.

Intervention characteristics

Study	Study design	Number of participants	Grade	Duration	Person implementing	Type of intervention
1. Abbott & Berninger (1999); random assignment; treatment fidelity: yes	Treatment-comparison (multiple treatments)	20 (struggling readers)	4th–7th	16 (1/week; 60 mina)	Researcher	Word study
2. Alfassi (1998); quasiexperimental; treatment fidelity: NR	Treatment-comparison	75 (struggling readers)	9th	20 sessions (daily × 4 weeks; 45 min)	Teacher	Comprehension
3. Allinder, Dunse, Brunken, & Obermiller-Krolikowski (2001); random assignment; treatment fidelity: yes	Treatment-comparison (multiple treatments)	49 (LD and struggling readers)	7th	30 sessions (3/week)	Teacher	Fluency
4. Anderson, Chan, & Henne (1995); random assignment; treatment fidelity: NR	Treatment-comparison	17 (struggling readers)	6th	70 sessions (daily × 14 weeks; 120 min)	Researcher	Comprehension
5. Bhat, Griffin, & Sindelar (2003)	Treatment-comparison	40 (LD)	6th–8th	18 sessions (3 days/week, 2 sessions/day)	Teacher	Word study
6. Bhattacharya & Ehri (2004); random assignment; treatment fidelity: NR	Treatment-comparison	60 (struggling readers)	6th–9th	4 sessions (daily 4 days; 30 min)	Researcher	Word study
7. Chan (1996); random assignment; treatment fidelity: NR	Treatment-comparison (multiple treatments)	40 (struggling readers)	7th	9 sessions (across 3 weeks; 60 min)	Researcher	Comprehension
8. DiCecco & Gleason (2002); random assignment; treatment fidelity: yes	Treatment-comparison (multiple treatments)	24 (LD)	6th–8th	20 sessions (daily 4 weeks; 40 min)	Researcher and teacher	Comprehension
9. L. S. Fuchs, Fuchs, & Kazdan (1999); quasiexperi-mental; treatment fidelity: yes	Treatment-comparison	102 (LD, struggling readers, MMR)	9th	40 sessions (5 every 2 weeks)	Teacher	Multicomponent (comprehension and fluency)
10. Hasselbring & Goin (2004); NR; treatment fidelity: NR	Treatment-comparison	125 (RD and struggling readers)	6th–8th	NR (daily; 30 min)	NR	Multicomponent (comprehension, word study)
11. Jitendra, Hoppes, & Xin (2000); random assignment; treatment fidelity: yes	Treatment-comparison	33 (LD and struggling readers)	6th–8th	15 sessions (daily 15 days; 30–40 min)	Researcher	Comprehension
12. Klingner & Vaughn (1996); random assignment; treatment fidelity: NR	Treatment-comparison (multiple treatments)	26 (LD, ESL)	7th–8th	27 sessions (daily; 35–40 min)	Researcher	Comprehension
13. Mastropieri et al. (2001); quasi-experimental; treatment fidelity: yes	Treatment-comparison	24 (LD and MR)	7th	25 sessions	Teacher	Multicomponent (comprehension and fluency)
14. Moore & Scevak (1995); random assignment; treatment fidelity: NR	Treatment-comparison	21 (struggling readers)	NR (high school)	NR (7 weeks)	Teacher	Comprehension
15. Penney (2002); quasi-experimental; treatment fidelity: yes	Treatment-comparison	33 (struggling readers)	NR (17-year-olds)	18 sessions (56 min)	Teacher	Word study
16. Wilder & Williams (2001); quasi-experimental; treatment fidelity: yes	Treatment-comparison (multiple treatments)	91 (LD)	6th–8th	NR (3/week; 45 min)	Teacher	Comprehension
17. Williams, Brown, Silverstein, & deCani (1994); random assignment; treatment fidelity: yes	Treatment-comparison (multiple treatments)	93 (LD)	7th–8th	NR (4 weeks; 40 min)	Teacher	Comprehension
18. Bryant et al. (2000); treatment fidelity: yes	Single group	14 (RD)	6th	NR (4 months; 90 min)	Teacher	Multicomponent (word identification, fluency, comprehension)
19. MacArthur & Haynes (1995); treatment fidelity: NR	Single group	10 (LD)	9th–10th	2 sessions	Researchers	Comprehension
20. Mercer, Campbell, Miller, Mercer, & Lane (2000); treatment fidelity: NR	Single group	49 (LD)	6th–8th	NR (daily; 5–6 min)	Teacher	Fluency
21. Daly & Martens (1994); treatment fidelity: yes	Single subject	2 (LD)	NR (11.11 yr.)	21 (1–2/day)	Teacher and researchers	Fluency
22. Freeland, Skinner, Jackson, McDaniel, & Smith (2000); treatment fidelity: yes	Single subject	3 (LD)	7th–8th, 11th	NR (daily)	Researcher	Fluency
23. Gardhill & Jitendra (1999); treatment fidelity: yes	Single subject	6 (LD)	6th and 8th	NR (14–20 weeks; 40–50 min)	NR	Comprehension
24. Lauterbach & Bender (1995); treatment fidelity: NR	Single subject	3 (LD, MMR)	9th	NR	Teacher	Comprehension
25. Scott & Shearer-Lingo (2002); treatment fidelity: yes	Single subject	3 (RD, EBD)	7th	NR	Researcher	Multicomponent (word study and fluency)
26. Steventon & Frederick (2003); treatment fidelity: yes	Single subject	1 (struggling reader)	NR (15-year-old)	26 sessions	Teacher	Multicomponent (word study, fluency)
27. Strong, Wehby, Falk, & Lane (2004); treatment fidelity: yes	Single subject	6 (RD)	7th–8th	NR (10–15 min/session, per program)	Researcher	Multicomponent (word study, fluency)
28. Vallecorsa & deBettencourt (1997); treatment fidelity: NR	Single subject	3 (LD)	7th	6 sessions (30 min)	Teacher	Comprehension
29. Valleley & Shriver (2003); treatment fidelity: yes	Single subject	4 (LD)	9th–10th	30 sessions (3/week for 10 weeks; 20 min)	Researcher	Fluency

Note. NR = not reported; LD = learning disability; MMR = mild mental retardation; MR = mental retardation; RD = reading disability; ESL = English as a Second Language; EBD = emotional or behavioral disability.

^aIndicates the number of minutes per session.

TABLE 2.

Outcomes by intervention type and design

Intervention	Measure	Findings/Results
Comprehension: T-C
Alfassi (1998)  T (reciprocal teaching): Working in small groups, students read text aloud, generated questions, summarized the text for their peers, discussed and clarified difficulties, and made predictions (n = 53).  C (current practices): School’s typical remedial reading instruction consisting of skills acquisition (n = 22).	Comprehension questions on taught passages Gates-MacGinitie Vocabulary subtest (standardized) Gates-MacGinitie Comprehension subtest (standardized)	T vs. C ES = 1.04a (p < .05) T vs. C ES = 0.16a (ns) T vs. C ES = 0.35a (ns)
Anderson, Chan, & Henne (1995)  T (strategy instruction): Four-phase instructional cycle that included (a) previewing, text reading, and comprehension monitoring; (b) analyzing text type and structure; (c) writing related to reading using text structure facilitators; and (d) generating questions and researching answers to enhance writing (n = 10).  C (current practices): School’s typical reading instruction (n = 7).	SAT Comprehension (standardized) Summaryb Wonderings (no. and complexity of questions generated)	T vs. C ES = 1.16a (p < .05) T vs. C ES = 2.08a (p < .05) T vs. C ES = 0.80a (ns)
Chan (1996)  T1 (reading strategy plus successive attributional training): Instruction in a clustering-rehearsal strategy on a sort-recall task (nonreading) before combining self-questioning strategy instruction with attributional training on a reading task. Attributional training involved having students compare pre- and posttest results and attributing (aloud) the improvement to the strategy (n = 11).  T2 (reading strategy plus simultaneous attributional training): A sort-recall task with no clustering-rehearsal strategy followed by the combined self-questioning strategy and attributional training on the reading task (n = 9).  T3 (attributional training only): Attributional training in the use of the clustering-rehearsal strategy on the sort-recall task (n = 11).  T4 (strategy training only): Strategy training in both clustering-rehearsal and self-questioning without attributional training (n = 9).	Short-answer comprehension test (no. correct)	T1 vs. T2 ES = 1.34a T1 vs. T3 ES = 1.68a T1 vs. T4 ES=1.50a T2 vs. T3 ES = 0.34a T3 vs. T4 T2 vs. T4 ES = .16*
DiCecco & Gleason (2002)  T (graphic organizers): Direct instruction using a graphic organizer of concept relationships (n = 12).  C (no graphic organizer): Instruction in the same content using guided discussions and note taking (n = 12).	Multiple-choice content knowledge test Fact recall Number of relational knowledge statements essays	ES = −0.18* T vs. C ES = 0.50 (ns) T vs. C ES = 0.08 (ns) T vs. C ES = 1.68 (p < .01)
Jitendra, Hoppes, & Xin (2000)  T (main idea): Main-idea strategy instruction using prompt cards and self-monitoring (n = 18).  C (current practices): School’s typical reading instruction (n = 15).	Main idea: Trained passages (identification/production of main idea statements) Main idea: Near transfer (similar narrative passages) Main idea: Far transfer (expository passages)	T vs. C ES =2.23a T vs. C ES = 2.57a T vs. C ES = 1.84a
Klingner & Vaughn (1996)  T1 (reciprocal teaching + tutoring): Reciprocal teaching plus peer tutoring on comprehension strategies (n = 13).  T2 (reciprocal teaching + cooperative learning): Reciprocal teaching plus strategy practice in cooperative learning groups (n = 13).	Gates MacGinitie Comprehension subtest (standardized) Passage comprehension test (% correct)	T1 vs. T2 ES = −1.42a T1 vs. T2 ES = 0.35a
Moore & Scevack (1995)  T (SLIC [summarize, link, image, check]): Explicit instruction in a set of strategies-summarize text, link text and visual aids, visually depict the relationship(s), and check for understanding (n = 11).  C (current practices): School’s typical reading instruction (n = 10).	Free recall (no. of details) Free recall (no. of main ideas) Multiple-choice comprehension test Transfer: Free recall details Transfer: Free recall main idea Transfer: Multiple-choice test	T v. C ES = −0.57 T vs. C ES = 0.07 T vs. C ES = −0.37 T vs. C ES = −0.39 T vs. C ES = −0.56 T vs. C ES = −0.36
Wilder & Williams (2001)  T1 (theme identification): Scaffolded instruction that included a prereading discussion, reading the story, postreading discussions guided by organizing questions, identifying the story theme, and relating the theme to real-life experiences (n = 47).  T2 (story comprehension): Comprehension instruction emphasizing vocabulary and plot through teacher-generated questions and discussion (n = 44).	Transfer: Story details in novel text (no. recalled) Transfer: Story components in novel text (main ideas) Theme concepts (understanding explicitly taught themes) Theme identification Theme application Vocabulary definitions Using vocabulary in sentences	T1 vs. T2 ES = 0.41 (ns) T1 vs. T2 ES = 0.59 (ns) T1 vs. T2 ES = 1.68 (p < .05) T1 vs. T2 ES = 5.93 (p < .01) T1 vs. T2 ES = 1.74 (p < .01) T1 vs. T2 ES = −0.25 T1 vs. T2 ES = −0.55
Williams, Brown, Silverstein, & deCani (1994)  T1 (themes instruction): Scaffolded instruction in prereading discussion, reading the story, participating in discussions guided by organizing questions, identifying the story theme, and relating that theme to real-life experiences (n = 53).  T2 (basal reading instruction): Instruction on the same content using a basal reader series adapted to the structure of prereading discussion, vocabulary development, story reading, and postreading discussion (n = 40).	Theme concept (understanding explicitly taught theme) Theme identification Theme application	T1 v. T2 ES = 1.41 (p < .001) T1 vs. T2 ES = 2.08a (p < .001) T1 v. T2 ES = 2.95a
Comprehension: Single group
MacArthur & Haynes (1995)  T (SALT [Student Assistance for Learning from Text]): Hypermedia versions of textbooks that provided either basic word recognition/decoding and vocabulary support or an enhanced version with additional support (question windows, glossary, teacher comments, and speech synthesis) for comprehending expository text (n = 10).	Short-answer and matching comprehension test	Tenhanced vs. Tbasic ES = 0.88 (not converted) (p < .05)
Comprehension: Single subject
Gardhill & Jitendra (1999)  T (Advanced story map construction): Explicit instruction in story grammar elements; phases included model, lead, and independent practice (n = 6).	M1: Story retell		PND (%)
		Student	M1	M2
		Marvin	100	63
		Mark	100	25
	M2: Basal comprehension test	Chad	100	25
		Mitch	100	13
		Tara	100	88
		Jack	100	100
Lauterbach & Bender in ref (1995)  T (read, ask and paraphrase strategy): Students taught to read the paragraph, identify the main idea and two details, and rewrite them in their own words (n = 3).	Paraphrasing (% correct)
				PND (%)
		A		91
		B		92
	Multiple-choice comprehension test (seventh-, eighth- and ninth-grade-levele materials)	C		100
			Seventh	Eighth		Ninth
		A	33	0		33
		B	100	100		100
Vallecorsa & deBettencourt (1997)  T (story mapping): Explicit instruction in eight story elements (definitions and multiple examples) and depicting story elements on a story map (n = 3).	Retell (no. of story elements included in retell)	C	0	0		0
		Student		PND (%)
		David		67
		Jason		100
		Nick		83
Fluency: T-C
Allinder, Dunse, Brunken, & Obermiller-Krolikowski (2001)  T (fluency strategy instruction): Students taught to focus on using one or more of the following strategies during a read-aloud conference: reading with inflection, self-monitoring for accuracy, reading at an appropriate pace, watching for word endings, and finger tracking (n = 33).  C (no strategy instruction): Students encouraged to do their best while reading aloud (n = 16).	WJRM Word Identification (standardized) WJRM Word Attack (standardized) WJRM Comprehension (standardized) Slope on maze task	T vs. C ES = −0.02 T vs. C ES = 0.08 T vs. C ES = −0.03 T vs. C ES = 0.79
Fluency: Single group
Mercer, Campbell, Miller, Mercer, & Lane (2000)  T1 (Great Leaps reading program): Instruction in sight phrases and oral reading with graphing of oral reading fluency for 19–25 months (n = 11).  T2: T1 for 10–18 months (n = 19).  T3: T1 for 6–9 months (n = 19).	CBM oral reading fluency	T1 ES = 0.37c T2 ES = 0.13 T3 ES = 0.24
Fluency: Single subject
Daly & Martens (1994)  T1 (subject passage preview): Student read a passage with feedback from instructor (n = 2).  T2 (taped words): Student read along with an audio-taped word list (n = 2).  T3 (listening passage preview): Student listened to an audio-taped passage while following along silently (n = 2).	Words read correctly per minute: Passage		PND (%)
			T1	T2	T3
	Words read correctly per minute: Word list	S3	100	57	57
		S4	100	57	14
		S3	43	29	29
		S4	57	86	71
Freeland, Skinner, Jackson, McDaniel, & Smith (2000)  T (repeated reading): Repeated oral passage reading (two reads) with error correction by the teacher (n = 3).  Baseline (silent reading): Silent passage reading (two reads) (n = 3).	CBM comprehension questions	Student		PND (%)d
	CBM fluency rate	Jason		71
		Bill		29
		Chris		86
		Jason		14
		Bill		57
		Chris		57
Valleley & Shriver (2003)  T (repeated readings): Engage in repeated readings in which the student rereads the same passage until he or she exhibits three consecutive fluency improvements.	Comprehension questions (no. correct, n = 10)	S1		6–10
		S2		4–10
	Oral reading fluency	S3		6–10
				PND (%)
		S1		12
		S2		24
		S3		17
Word study: T-C
Abbott & Berninger (1999)  T1 (structural analysis): Instruction in the alphabetic principle, phonological decoding (applied phonics and structural analysis), structural analysis focused on affixes and suffixes, and repeated oral reading with error correction (using structural analysis) and comprehension monitoring (n = 10).  T2 (study skills): T1 with synthetic phonics strategies (i.e., letter-sound correspondence) and study skills instruction (workbook pages on note taking, outlining, and paragraph writing) in place of structural analysis instruction and application (n = 10).	WRMT-R: Comprehension (standardized) WRMT-R Word Identification (standardized) WRMT-R Word Attack (standardized) Qualitative Reading Inventory (standardized) TOWRE (standardized) TOWRE Pseudowords (standardized)	T1 vs. T2 ES = −0.12a (ns) T1 vs. T2 ES = −0.17a (ns) T1 vs. T2 ES = −0.08a (ns) T1 vs. T2 ES = 0.19a (ns) T1 vs. T2 ES = −0.31a (ns) T1 vs. T2 ES = 0.04a (ns)
Bhat, Griffin, & Sindelair (2003)  T (Great Leaps reading program + phonemic awareness): Phonological and phonemic awareness lessons from Great Leaps reading program supplemented with additional phonemic awareness activities, including phoneme blending, segmenting, reversal, and substitution (n = 20). C (current practices): School’s typical reading instruction (n = 20).	CTOPp (standardized) WRMT Word Identification (standardized)	T vs. C ES = 1.59a (p < .001) T vs. C ES = 0.15a (ns)
Bhattacharya & Ehri (2004)  T1 (syllable chunking strategy): Students were taught to orally divide multisyllabic words into syllables, state the number of syllables, match them to their spelling, and blend the syllables to say the whole word. Corrective feedback was provided after each step (n = 20).  T2 (whole-word reading): Students practiced reading multisyllabic words with no applied strategy. Corrective feedback was provided (n = 20).  C1 (current practices): School’s typical reading instruction (n = 20).	WRMT-R Word Attack (standardized) Syllable segmentation Decoding words Decoding subtle misspellings Decoding pseudowords by analogy	T1 vs. C ES = 1.40 T2 vs. C ES = 0.43 T1 vs. C ES = 1.14 T2 vs. C ES = 0.20 T2 vs. C ES = 0.20 T1 vs. C ES = 0.65 T2 vs. C ES = 0.42 T1 vs. C ES = 1.14 T2 vs. C ES = 0.51 T1 vs. C ES = 0.50
Penney (2002)  T (phonemic decoding): Students read aloud from text; words read slowly or incorrectly were then taught using the Glass analysis method of rehearsing the pronunciation of letter sequences that form pronounceable parts of words (n = 21).  C (current practices): School’s typical reading instruction (n = 11).	PPVT (standardized) WRMT Word Identification (standardized) WRMT Word Attack WRMT Passage Comprehension	T2 vs. C ES = 0.03 T vs. C ES = 0.82 T vs. C ES = 0.48a (p < .001) T vs. C ES = 0.43a (p < .05) T vs. C ES = 0.65a (p < .001)
Multicomponent: T-C
L. S. Fuchs, Fuchs, & Kazdan (1999)  T (peer-assisted learning strategies [PALS]): Partner reading, paragraph shrinking, and prediction relay implemented using a dyadic structure (n = 52).  C (current practices): School’s typical reading instruction with no peer-mediated learning activities (n = 50).	CRAB Oral Reading Fluency (standardized) CRAB Comprehension	T vs. C ES = 0.05 T vs. C ES = 0.31
Hasselbring & Goin (2004)  T (computer-based literacy instruction): Instruction in Peabody Literacy Lab: Reading Lab, with videos to support students in building mental models from text; Word Lab, with practice reading words on timed tasks; Spelling Lab, with practice typing a word that is pronounced, broken into parts, and used in a sentence plus additional spelling fluency practice (n = 63).  C (current practices): School’s typical reading instruction (n = 62).	SDRT Comprehension (standardized) SDRT Auditory Vocabulary (standardized) SDRT Phonetic Analysis (standardized) SDRT Structural Analysis (standardized)	T vs. C ES = 1.00a T vs. C ES = 0.75a T vs. C ES = 0.23a T vs. C ES = 0.44a
Mastropieri et al. (2001)  T (peer tutoring condition): Partner reading with error correction, passage summarization (“Get the Gist”), and questioning strategies for during and after reading implemented using same-age peer tutoring sessions (n = 12).  C (current practices): School’s typical reading instruction (n = 12).	Open-ended comprehension test	T vs. C ES = 1.18 (p < .05)
Multicomponent: Single group
Bryant et al. (2000)  T (collaborative strategic reading [CSR] + word reading strategy and fluency): Instruction in the four main components of CSR-predicting, word learning strategies (e.g., using context clues), finding the main idea, and summarizing-plus a word identification strategy (DISSECT) and structured partner reading (n = 14).	Word identification test of oral reading fluency (standardized) Jamestown Timed Reading Passage Comprehension questions (no. correct)	T ES = 0.64 (unconverted) (p < .05) T ES = 0.67 (p < .05) T ES = 0.22
Multicomponent: Single subject
Scott & Shearer-Lingo (2002)  T1 (Teach Your Child): Phonics instruction with teacher modeling of letter-sound relationships and opportunities for guided practice (n = 3).  T2 (Great Leaps reading program): 1-min timings of letter sounds, sight phrases, and story reading with teacher monitoring (n = 3).	Oral reading fluency		PND (%)
		Student	T1	T2
		Tony	100	100
		Billy	100	95
		John	0	100
Steventon & Frederick (2003)  T (corrective reading + repeated reading): Explicit decoding instruction using the Corrective Reading program plus repeated reading (n = 1).	Oral reading fluency		PND (%)
		Carl: Practiced text	54
		Carl: Novel text	8
Strong, Wehby, Falk, & Lane (2004)  T1 (corrective reading): Instruction in decoding strategies through lessons that consist of word attack skills, group reading, and workbook exercises (n = 6).  T2 (corrective reading + repeated reading): T1 plus partner reading with teacher-provided corrective feedback during first two reads and partner-provided feedback for the subsequent read (n = 6).	SRA probes (words correct per minute)		PND (%)
		Student	T1	T2
		Jim	29	92
		Dave	29	100
		Joe	55	83
		Mike	45	100
		Steve	93	100
		Jay	40	75

Note. T = treatment; C = comparison; ES = effect size; PND = percentage of nonoverlapping data; SAT = Stanford Achievement Test; WJRM = Woodcock Johnson Reading Mastery; CBM = curriculum-based measure; WRMT = Woodcock Reading Mastery Test; WRMT-R = Woodcock Reading Mastery Test-Revised; TOWRE = Test of Word Reading Efficiency; CTOPp = Comprehensive Test of Phonological Processing; PPVT = Peabody Picture Vocabulary Test; CRAB = Comprehensive Reading Assessment Battery; SDRT = Stanford Diagnostic Reading Test; SRA = Science Research Associates.

^aIndicates effect size adjusted for pretest differences.

^bAll measures are researcher developed unless indicated by a parenthetical note (e.g., standardized).

^cRepeated measures effect size converted to the metric of Cohen’s d.

^dPND calculated as the percentage nonoverlapping data between the baseline and treatment instructional conditions for each student.

Effect size calculation

Effect sizes were calculated for studies that provided adequate information. For studies lacking data necessary to compute effect sizes, data were summarized using findings from statistical analyses or descriptive statistics. For treatment–comparison design studies, the effect size, d, was calculated as the difference between the mean posttest score of the participants in the intervention condition minus the mean posttest score of the participants in the comparison condition divided by the pooled standard deviation. For studies in this synthesis that employed a treatment–comparison design, effect sizes can be interpreted as d = 0.20 is small, d = 0.50 is medium, and d = 0.80 is a large effect (Cohen, 1988). Effects were adjusted for pretest differences when data were provided. For single-group studies, effect sizes were calculated as the standardized mean change (Cooper, 1998). Outcomes from single-subject studies were calculated as the percentage of nonoverlapping data (PND) (Scruggs, Mastropieri, & Casto, 1987). PND is calculated as the percentage of data points during the treatment phase that are higher than the highest data point from the baseline phase. PND was selected because it offered a more parsimonious means of reporting outcomes for single-subject studies and provided common criteria for comparing treatment impact.

Results

Data Analysis Plan

A range of study designs and intervention types was represented in this synthesis. To fully explore the data, we conducted several types of analyses. First, we synthesized study features (e.g., sample size and study design) to highlight similarities, differences, and salient elements across the corpus of studies. Second, we conducted a meta-analysis of a subset of treatment–comparison design studies to determine the overall effect of reading interventions on students’ reading comprehension. In addition to an overall point estimate of reading intervention effects, we reported effects on comprehension by measurement and intervention type. Last, we synthesized trends and results by intervention type across all studies, including single-group and single-subject design studies.

Study Features

A total of 29 intervention studies, all reported in journal articles, met our criteria for inclusion in the synthesis. Studies appeared in a range of journals (as can be seen in the reference list) and were distributed relatively evenly across the years of interest (1994 to 2004). Each study’s design and sample characteristics are described in Table 1. In the following sections, we summarize information on study features, including sample characteristics, design, and duration of the intervention as well as fidelity of implementation.

Sample characteristics

The 29 studies included 976 students. Sample sizes ranged from 1 to 125, with an average of 51 participants for treatment–comparison studies. The majority of studies targeted middle school students (n = 19). Five studies focused on high school students, 2 on both middle and high school students, and 3 reported only students’ ages. Although our criteria included interventions for all struggling readers, including those without identified disabilities, only 8 studies included samples of struggling readers without disabilities. The other studies included students with learning or reading disabilities (n = 17) or a combination of both students with and without disabilities (n = 4).

Study design

The corpus of studies included 17 treatment–comparison, 9 single-subject, and 3 single-group design studies. The distribution of intervention type by design is displayed in Table 3. The number of treatment–comparison studies with specific design elements that are characteristic of high quality studies (Institute of Education Sciences, 2003; Raudenbush, 2005; Shadish, 2002) is indicated in Table 4. The three elements in Table 4 were selected because they strengthen the validity of study conclusions when appropriately employed. As indicated, only 2 studies (Abbott & Berninger, 1999; Allinder, Dunse, Brunken, & Obermiller-Krolikowski, 2001) randomly assigned students to conditions, reported implementation fidelity, and measured student outcomes using standardized measures.

TABLE 3.

Type of intervention by study design

	Study design
Intervention type	Treatment–comparison	Single group	Single subject	Marginal totals
Comprehension	9	1	3	13
Fluency	1	1	3	5
Word study	4	0	0	4
Multicomponent	3	1	3	7
Marginal totals	17	3	9	29

TABLE 4.

Quality of treatment–comparison studies

Element	Number of studies
Random assignment to conditions	10
Fidelity of treatment reported	9
Standardized dependent measures	10
Random assignment, treatment fidelity, and standardized measures	2

Intervention design and implementation

The number of intervention sessions ranged from 2 to 70. For 11 studies, the number of sessions was not reported and could not be determined from the information provided. Similarly, the frequency and length of sessions was inconsistently reported but is provided in Table 1 when available. For studies that reported the length and number of sessions (n = 12), students were engaged in an average of 23 hr of instruction. For treatment–comparison design studies, the average number of instructional hours provided was 26 (n = 10).

Narrative text was used in most text-level interventions (n = 12). Two studies used both narrative and expository text during the intervention, and 7 used expository text exclusively. For 4 studies, the type of text used was not discernable, and as would be expected, the word-level studies did not include connected text. About an equal number of study interventions was implemented by teachers (n = 13) and researchers (n = 12). Two interventions were implemented by both teachers and researchers, and the person implementing the intervention could not be determined from 2 studies.

Meta-Analysis

To summarize the effect of reading interventions on students’ comprehension, we conducted a meta-analysis of a study subset (k = 13; Abbott & Berninger, 1999; Alfassi, 1998; Allinder et al., 2001; Anderson, Chan, & Henne, 1995; DiCecco & Gleason, 2002; L. S. Fuchs, Fuchs, & Kazdan, 1999; Hasselbring & Goin, 2004; Jitendra, Hoppes, & Xin, 2000; Mastropieri et al., 2001; Moore & Scevak, 1995; Penney, 2002; Wilder & Williams, 2001; Williams, Brown, Silverstein, & deCani, 1994). Studies with theoretically similar contrasts and measures of reading comprehension were included in the meta-analysis. All selected studies compared the effects of a reading intervention with a comparison condition in which the construct of interest was absent. By selecting only studies with contrasts between a treatment condition and a no-treatment comparison condition, we could ensure that the resulting point estimate of the effect could be meaningfully interpreted.

The majority of qualifying studies reported multiple comprehension dependent variables. Thus, we first calculated a composite effect for each study using methods outlined by Rosenthal and Rubin (1986) such that each study contributed only one effect to the aggregate. In these calculations, effects from standardized measure were weighted more heavily (w = 2) than effects from research-developed measures. We analyzed a random-effects model with one predictor variable (intervention type) to account for the presence of unexplained variance and to provide a more conservative estimate of effect significance. A weighted average of effects was estimated and the amount of variance between study effects calculated using the Q statistic (Shadish & Haddock, 1994). In addition to an overall point estimate of the effect of reading interventions, we also calculated weighted averages to highlight effects of certain intervention characteristics (e.g., using narrative versus expository text). When reporting weighted mean effects, only outcomes from studies with treatment–comparison conditions were included. Effects from single-group studies were excluded because only one study (Mercer, Campbell, Miller, Mercer & Lane, 2000) provided the information needed to convert the repeated-measures effect size into the same metric as an independent group effect size.

Overall effect on comprehension

The 13 treatment–comparison studies were included in the meta-analysis because they (a) had theoretically similar contrasts and measures of reading comprehension and (b) examined the effects of a reading intervention with a comparison in which the construct of interest was absent. In 8 studies, the contrast was between the intervention of interest and the school’s current reading instruction. In 5 studies, the comparison condition also received an intervention, but the construct or strategy of interest was absent from that condition. The remaining 4 treatment–comparison studies in the synthesis were eliminated from the meta-analysis because they did not include a comprehension measure (Bhat, Griffin, & Sindelair, 2003; Bhattacharya & Ehri, 2004) or they did not include a no-treatment comparison condition (Chan, 1996; Klingner & Vaughn, 1996).

A random-effects model was used to provide a more conservative estimate of intervention effect significance. In this model, the weighted average of the difference in comprehension outcomes between students in the treatment conditions and students in the comparison conditions was large (effect size = 0.89; 95% confidence interval (CI) = 0.42, 1.36). That is, students in the treatment conditions scored, on average, more than two thirds of a standard deviation higher than students in the comparison conditions on measures of comprehension, and the effect was significantly different from zero.

To examine whether researcher-developed or curriculum-based measures inflated the effect of reading interventions, we also calculated the effect based on standardized measures only. For this analysis, seven studies were included; the other six studies were eliminated from this secondary analysis because they did not include a standardized measure of comprehension. When limited to only studies that included a standardized measure of comprehension, the random-effects model yielded a moderate average effect (effect size = 0.47; 95% CI = 0.12, 0.82). The effect of reading interventions on comprehension was quite large (effect size = 1.19; 95% CI = 1.10, 1.37) when researcher-developed measures were used to estimate the effect (k = 9).

In a fixed-effects model, intervention type was a significant predictor of effect size variation (Q_between = 22.33, p < .05), which suggests that the effect sizes were not similar across the categories. Weighted average effects for each intervention type (comprehension, fluency, word study, and multicomponent) were calculated and are presented in Table 5. For fluency and word study interventions, the effect was not significant—the average effect on comprehension was not different from zero. For the other intervention types, the effect was significantly different from zero but differed in magnitude. Bonferroni post hoc contrasts showed a significant difference in effects on comprehension between comprehension and multicompo-nent interventions (p < .025). There was no significant difference between the effects of word study interventions and multicomponent interventions (p > .025).

TABLE 5.

Average weighted effects by measurement and intervention type

	Effect size (95% confidence interval)
Measurement type
All measures (n = 13)	0.89 (0.42, 1.36)
Standardized measures (n = 7)	0.47 (0.12, 0.82)
Researcher developed measures (n = 9)	1.19 (1.10, 1.37)
Intervention Type
Fluency (n = 1)	−0.03 (−0.56, 0.62)
Word study (n = 2)	0.34 (−0.22, 0.88)
Multicomponent (n = 3)	0.72 (0.45, 0.99)
Comprehension (n = 7)	1.23 (0.96, 1.5)

We also computed weighted average effects for studies with common characteristics. Whether an intervention was implemented by the researcher (n = 4, average effect size = 1.15) or the students’ teacher (n = 8, effect size = 0.77), the effects were large. The 95% CIs for these two conditions did not overlap, suggesting that they are significantly different. Effects on comprehension were different depending on the student population. Moderate average effects were found for samples of struggling readers (n = 5, effect size = 0.45) or both struggling readers and students with disabilities (n = 4, effect size = 0.68), but a large effect (n = 4, effect size = 1.50) was found for studies with samples of only students with disabilities.

Eleven of the 13 studies included in the meta-analysis used reading of connected text as part of the intervention. In an analysis of studies that reported the type of text used, the weighted average effect for interventions using expository text was moderate (n = 3, effect size = 0.53), whereas the average effect for those focusing on narrative text was high (n = 6, effect size = 1.30). Closer examination of the studies with interventions focused on expository text (Alfassi, 1998; DiCecco & Gleason, 2002; Moore & Scevak, 1995) showed that two studies tested the effects of a multicomponent intervention similar in structure to reciprocal teaching and one examined the effects of using graphic organizers.

Intervention Variables

For this synthesis, we examined findings from treatment–comparison design studies first, because the findings from these studies provide the greatest confidence about causal inferences. We then used results from single-group and single-subject design studies to support or refute findings from the treatment–comparison design studies. Findings are summarized by intervention type. Intervention type was defined as the primary reading component addressed by the intervention (i.e., word study, fluency, vocabulary, comprehension). The corpus of studies did not include any vocabulary interventions but did include several studies that addressed multiple components in which vocabulary instruction was represented. Within each summary, findings for different reading outcomes (e.g., fluency, word reading, comprehension) are reported separately to highlight the interventions’ effects on component reading skills.

Comprehension

Nine treatment–comparison studies (Alfassi, 1998; Anderson et al., 1995; Chan, 1996; DiCecco & Gleason, 2002; Jitendra et al., 2000; Klingner & Vaughn, 1996; Moore & Scevak, 1995; Wilder & Williams, 2001; Williams et al., 1994) focused on comprehension. Among these studies, several (Alfassi, 1998; Anderson et al., 1995; Klingner & Vaughn, 1996; Moore & Scevak, 1995) examined interventions in which students were taught a combination of reading comprehension skills and strategies, an approach with evidence of effectiveness in improving students’ general comprehension (NRP, 2000; RAND Reading Study Group, 2002). Two studies (Alfassi, 1998; Klingner & Vaughn, 1996) employed reciprocal teaching (Palincsar, Brown, & Martin, 1987), a model that includes previewing, clarifying, generating questions, and summarizing and has been shown to be highly effective in improving comprehension (see for review, Rosenshine & Meister, 1994). Klingner and Vaughn (1996) reported mixed results when the grouping structure of a reciprocal teaching intervention was manipulated during student application and practice. On a standardized measure of comprehension, cooperative grouping was the more effective model (effect size = 1.42). On a researcher-developed comprehension measure, the effects were small but favored the peer tutoring group (effect size = 0.35). It is likely that the standardized test outcome is more reliable, suggesting greater effects from the use of cooperative grouping structures, at least for English language learners with reading difficulties. In another study, effects of reciprocal teaching on comprehension were moderate to high (effect size = 0.35 to 1.04; Alfassi, 1998) when implemented in a remedial high school setting, a context not typically examined in previous studies of reciprocal teaching (Alfassi, 1998).

The multiple-strategy intervention in Anderson et al. (1995) resulted in large effects (effect size = 0.80 to 2.08). The repertoire of strategies included previewing and using knowledge of text structure to facilitate understanding. However, another study (Moore & Scevak, 1995), which focused on teaching students to use text structure and features to summarize expository text, reported no effects (effect size = −0.57 to 0.07). It should be noted that the intervention provided in the Anderson and colleagues study (1995) was conducted for 140 hr (a very extensive intervention), and the amount of time for the intervention in the Moore and Scevak study (1995) was not specified, but the study was conducted for only 7 weeks—suggesting a significantly less extensive intervention.

Chan (1996) manipulated both strategy instruction and attribution training and found that poor readers benefited from some attribution training, with the most effective model being attribution training plus successive strategy training (effect size = 1.68). In addition, all three strategy conditions were more effective than the attribution-only condition, which suggests that poor readers also benefit from explicit strategy instruction.

Using graphic organizers is another strategy with demonstrated efficacy in improving comprehension (Kim et al., 2004). One experimental study (DiCecco & Gleason, 2002) and two single-subject studies (Gardhill & Jitendra, 1999; Vallecorsa & deBettencourt, 1997) examined the impact of teaching students to use graphic organizers. In DiCecco and Gleason (2002), the effect of a concept relationship graphic organizer intervention on relational statement production was large (effect size = 1.68). However, the effect was mixed for measures of content knowledge (effect size = 0.08 to 0.50). Other studies also indicated that graphic organizers assisted students in identifying information related to the organizer but were less effective in improving students’ overall understanding of text. For example, in a single-subject study of a story mapping intervention, Gardhill and Jitendra (1999) found mixed results on general comprehension questions (PND = 13% to 100%) but consistent improvement compared to baseline on story retell (PND = 100%). Similarly, all three students in a study of explicit story mapping (Vallecorsa & deBettencourt, 1997) increased the number of story elements included in a retell (PND = 67% to 100%).

Other studies focused on a single comprehension strategy (Jitendra et al., 2000; Wilder & Williams, 2001; Williams et al., 1994). Studies of single-strategy interventions showed large effects on measures aligned closely with the intervention but limited examples of transfer to more general comprehension measures. For example, students who were taught to identify main ideas within text outperformed students in the comparison condition on a task of identifying and producing main idea statements (effect size = 2.23; Jitendra et al., 2000). Although the treatment effects were maintained on near and far transfer measures (effect size = 1.84 to 2.57), scores decreased significantly for both conditions on transfer passages, indicating a lack of transfer to novel contexts. Similarly, interventions in which students were taught to identify and apply story themes (Wilder & Williams, 2001; Williams et al., 1994) resulted in large effects on measures of theme identification and application (effect size = 1.41 to 5.93). Effects of this intervention on general comprehension tasks were somewhat attenuated, although still demonstrating moderate effects (effect size = 0.41 to 0.59; Wilder & Williams, 2001).

Three studies included information about students’ decoding abilities (Alfassi, 1998; DiCecco & Gleason, 2002; Jitendra et al., 2000). In all three studies, students were adequate decoders but poor comprehenders. The average effect of the comprehension interventions was large (effect size = 1.04).

Multicomponent

Studies (L. S. Fuchs et al., 1999; Hasselbring & Goin; 2004; Mastropieri et al., 2001) were classified as multicomponent when the interventions included instruction in more than one component of reading, such as word study with fluency or fluency with comprehension. Two multicomponent studies (L. S. Fuchs et al., 1999; Mastropieri et al., 2001) featured a slightly modified version of a peer-assisted learning comprehension and fluency intervention, an instructional model with demonstrated efficacy in the early elementary grades (D. Fuchs, Fuchs, Mathes, & Simmons, 1997). Results when using this intervention model with older struggling readers were mixed. When implemented in an inclusive setting on a biweekly basis, effects on comprehension skills were small (effect size = 0.31; L. S. Fuchs et al., 1999) yet were quite large when implemented daily in a self- contained resource room (effect size = 1.18; Mastropieri et al., 2001). It should be noted that the large effect size was computed from data on a researcher-developed measure, whereas the smaller effect was based on data from a standardized measure, which is a more reliable measure of the intervention’s effect.

In a single-group design study (Bryant et al., 2000), students participated in an enhanced collaborative strategic reading intervention during which they applied word learning, word reading, and comprehension strategies and practiced fluent reading. This was the only study that examined the effects of an instructional model with all four components included. Effects on word identification and oral reading fluency were moderate (effect size = 0.64, effect size = 0.67, respectively), but effects on comprehension were small (effect size = 0.22).

Hasselbring and Goin (2004) implemented a computer-based intervention that provided students with word reading and spelling practice and comprehension support during text reading. Effects on comprehension (effect size = 1.0) and vocabulary (effect size = 0.75) were large. Effects on word-level skills, however, were small (effect size = 0.23 to 0.44). Results from a single-subject design study with word study as one instructional component (Strong, Wehby, Falk, & Lane, 2004), indicated more consistent improvement in students’ oral reading fluency when word study was combined with fluency practice than when word study instruction alone was provided. However, Steventon and Frederick (2003) had less success with one student who participated in a similar word study and fluency intervention. Their results showed less improvement compared to baseline for oral reading fluency and virtually no transfer of fluent reading to novel text.

There were only two studies that featured technology prominently in the instruction. One was the previously discussed multicomponent intervention by Hasselbring and Goin (2004). The other was a study that used computers to enhance text and support reading (MacArthur & Haynes, 1995), which yielded an effect size in favor of basic text support (word recognition and decoding with vocabulary support) when compared with enhanced text support (additional support that includes question windows, glossary, teacher comments, and speech synthesis) for comprehending expository text.

Fluency

The synthesis included one treatment–comparison design study of fluency (Allinder et al., 2001). Allinder et al. (2001) studied the effects of prompting students to use strategies for fluent reading (e.g., reading with inflection) and found no effects on standardized word-level or comprehension measures. The other studies of fluency focused on improving oral reading fluency, often through word or phrase reading fluency and/or repeated reading. Results were mixed with inconsistent improvements in oral reading fluency compared to baseline (Freeland, Skinner, Jackson, McDaniel & Smith, 2000; Mercer et al., 2000; Valleley & Shriver, 2003).

Word study

Three of four experimental word-level studies examined the effects of advanced word reading strategies (Abbott & Berninger, 1999; Bhattacharya & Ehri, 2004; Penney, 2002). The fourth (Bhat et al., 2003) studied the effects of a phonemic awareness intervention. Results of the phonemic awareness intervention were positive, with large effects on phonemic processing (effect size = 1.59). However, the overall effect of improved phonemic processing transferred minimally to improved word identification (effect size = 0.15).

Results for the three structural analysis studies were mixed, with effects ranging from −0.31 to 1.40. Bhattacharya and Ehri (2004) found that although having students practice whole-word reading versus providing no word reading instruction at all had a small effect (effect size = 0.43), teaching students a structural analysis approach (i.e., multisyllabic chunking) had a large effect (effect size = 1.40). In another study that compared a structural analysis approach to typical reading instruction, the effects on word reading were moderate (effect size = 0.43 to 0.48; Penney, 2002). In the third study (Abbott & Berninger, 1999), the effect of phonics and structural analysis instruction on word reading skills was minimal (effect size = −.31 to .04). However, in the latter study, the comparison and treatment conditions received identical interventions, with the exception of the decoding strategy taught: The comparison condition was taught a synthetic phonics strategy and the treatment condition a combination of phonics and structurally analysis. Results may have been lower in this study because, with both conditions being provided a fairly robust treatment, the contrasted conditions were not as dissimilar as in the other two studies.

Across studies, the weighted average effect of structural analysis instruction on word reading skills was moderate (effect size = .36, 95% CI = .03, .69). Two studies (Abbott & Berninger, 1999; Penney, 2002) measured comprehension as an outcome of a word-level intervention. Again, the results were mixed (effect size = −0.12 to 0.65).

Discussion

Results from the meta-analysis indicate that students with reading difficulties and disabilities can improve their comprehension when provided with a targeted reading intervention in comprehension, multiple reading components, or, to a lesser extent, word reading strategies. Even when using standardized measures, which offer a more generalized measure of comprehension, the effect is moderate, providing students with an average of a half standard deviation advantage compared to their peers without the treatment.

A primary finding from this synthesis is that struggling readers can improve in their reading comprehension when taught reading comprehension practices. Seemingly obvious, this phenomenon is quite significant because many struggling readers in older grades (6 through 12) are not provided effective instruction in reading comprehension. In fact, interventions that specifically targeted students with learning disabilities were associated with the highest gains in reading comprehension. Results from this synthesis suggest that explicit instruction in comprehension benefited students with reading difficulties and disabilities. Findings also suggest that there may be a diminishing relationship between accuracy (e.g., word recognition and fluent reading) and comprehension with secondary students. When students reach the upper elementary grades, other factors, such as background knowledge, word knowledge, and use of strategies, contribute to comprehension (Kintsch & Kintsch, 2004). The large effects of interventions that developed students’ strategy knowledge and use and the relatively lower effects of other types of interventions on comprehension support these previous findings. Thus, for students who lack word reading skills, it is necessary to build these word-level skills while teaching comprehension so that access to increasingly difficult levels of print is available to them.

As indicated by the meta-analysis, word-level interventions are associated with small to moderate effects on comprehension (d = .34). This supports some studies in early grade levels (e.g., Baumann et al., 2002) that found little effect on comprehension from structural analysis interventions. Although the average effect was not significantly different from zero, the small to moderate effect is an important finding, particularly for older students with very low decoding skills who require extensive instruction in word-level skills. It is valuable to know that there is a small to moderate effect for comprehension from word-level interventions.

The data trend from the studies of fluency indicates that increased reading rate and accuracy did not always result in improved comprehension (e.g., Allinder et al., 2001). These results support other research on the relationship between comprehension and fluency for older students. For example, Kuhn and Stahl (2003) found that although fluency instruction improved the processing skills that facilitate comprehension, few fluency interventions fostered better general comprehension. Stated more succinctly, as students improved their oral reading fluency, comprehension did not jointly improve. Others also report that the correlation between oral reading fluency and comprehension appears to be a developmental relationship, decreasing steadily with age and with text difficulty (Francis, Fletcher, Catts, & Tomblin, 2004; Paris, Carpenter, Paris, & Hamilton, 2004). For educators, the message from these findings is that “an intense focus on fluency may pay a short-term dividend, [however] the cost-benefit analysis of such an emphasis for adolescent learners looks less attractive” (Underwood & Pearson, 2004, p. 139).

Although we do not think the evidence from this synthesis would suggest forgoing instruction in reading skills such as fluency or advanced decoding strategies with secondary struggling readers—particularly for students whose word reading skills are exceedingly low—the findings from this synthesis do encourage educators to include instruction targeting comprehension skills. Results from this synthesis suggest that older struggling readers benefit from explicit comprehension strategy instruction—that is, modeling and thinking aloud how to self-question and reflect during and after reading and engaging students to become actively involved in monitoring their understanding and processing text meaning. This form of collaboration among students as they read and construct meaning has been well defined by Beck and colleagues in their work on “questioning the author” (Beck & McKeown, 2006; Beck, McKeown, Worthy, Sandora, & Kucan, 1997).

The moderate and large effects on training and near-transfer measures did not frequently generalize to measures of broader, more general comprehension. It appears that comprehension and multicomponent interventions can result in students’ becoming more proficient in applying learned strategies and learning taught content, but they often do not result in readers who use the strategies independently and flexibly in novel contexts. For example, Alfassi (1998) found that the significant effect for condition on researcher-developed measures (effect size = 1.04) did not generalize to standardized measures of broad comprehension and vocabulary skills (0.35 and 0.16, respectively). For single-strategy interventions, students were successful on measures related to the targeted strategy (e.g., identifying the main idea after explicit main idea instruction; Jitendra et al., 2000), but on broader measures of comprehension, effects were generally lower and less consistent. These results suggest that older struggling readers may need additional opportunities to apply newly learned strategies to novel text or may need to learn other practices related to text reflection, self-questioning, and engagement.

On the basis of the mixed results from studies that examined the effects of early reading instructional practices (e.g., reciprocal teaching and graphic organizers), we conclude that educators cannot assume that instructional practices with demonstrated efficacy in the lower grades will be equally as effective when implemented with older struggling readers. There are several possible explanations for this. First, the learning needs of this population may differ from those of younger students. Some of these students may have had extensive interventions addressing word-level skills and few interventions addressing practices for comprehending text. This may explain why comprehension interventions for students with learning disabilities were associated with exceedingly high effect sizes. It may be that students with disabilities have had relatively limited instruction in this area. Second, older readers are required to read more information or expository text. Although the number of expository text studies was few in this synthesis, overall narrative text was associated with higher effect sizes from comprehension interventions than expository text. Thus, comprehension practices developed to address narrative text comprehension may benefit narrative text comprehension and have a lower impact on reading expository text—at least for older struggling readers. It may also be that older struggling readers display reading difficulties that are more recalcitrant and require more intensive interventions (e.g., longer duration, more targeted) to achieve similar results.

Limitations

As with any synthesis, our findings are tempered by a few limitations. First, issues of measurement in the area of comprehension are extensive (Snow, 2003). Comprehension is a difficult construct to assess, and many of the studies measured comprehension in varied ways. Comprehension was measured by tasks that ranged from memorization activities (e.g., recall) to indications of complex cognitive behaviors (drawing inferences). Some theorists would argue that pooling or comparing outcomes from measures assessing a spectrum of skills may be misleading. Given the limited number of measures and the limited number of studies within each given category of skill complexity, however, we believed that gaining an understanding of the overall effect on comprehension provides a summary of what we know and insight into future research needed.

Second, the use of researcher-developed measures (or nonstandardized measures) was associated with higher effect sizes than standardized measures. This is a consistent finding from intervention research in education (e.g., Swanson, Hoskyn, & Lee, 1999) and should be considered when interpreting the results from intervention studies.

Finally, syntheses are only as good as the quality of the research articles available. We think that this synthesis yields valuable findings; however, only additional research and better-quality research will determine whether these findings will be supported over time.

Implications and Future Research

This synthesis yields several implications for educators. First, we think that these studies indicate that comprehension practices that engage students in thinking about text, learning from text, and discussing what they know are likely to be associated with improved comprehension outcomes for students with reading difficulties and disabilities. Second, the comprehension practices used are more effective for narrative text than expository text. We think that teachers may want to consider the use of additional elements, such as graphic organizers and calling students’ attention to text structures when students are reading relevant expository or information texts. Third, comprehension outcomes were higher when interventions were implemented by researchers in contrast to when implemented by teachers. Because it is likely that researchers are more attentive to implementing interventions with high levels of fidelity, teachers may want to consider their fidelity of implementation when targeting comprehension practices.

There are several important areas related to reading comprehension that this synthesis was unable to address and would be important to consider in future syntheses. As stated in the introduction, RAND Reading Study Group (2002) identified several critical elements that contributed to comprehension: the reader, the text, and the activity. This synthesis examined the extent to which students identified by previous researchers as having reading difficulties or disabilities could demonstrate improved comprehension when participating in specified interventions designed to improve their reading. There are many other key areas related to reading comprehension, including the relationship between the sociocultural context and the student, teacher, and setting. We think that these variables as well as social and affective variables related to students’ interest and motivation would make for valuable understanding of the role of context on students’ comprehension. This synthesis also did not examine the relationship between writing interventions on reading comprehension outcomes for older struggling readers. An extension of this synthesis may provide additional insight into effects of writing interventions on comprehension for struggling readers in middle and high school.

We also think that this synthesis provides ample support for additional research in the area of reading comprehension. Recently, a report on adolescent literacy indicated that as many as 70% of secondary students require some form of reading remediation (Biancarosa & Snow, 2004). The type of reading instruction required for this large number of secondary students is not well defined; however, we can be certain that many of these students will require effective instruction targeted at improving their reading comprehension. Future research addressing the needs of this varied group of struggling adolescent readers is needed, including improved measurement in reading comprehension; effective interventions for various text types, including information text; studies that improve our confidence of effectiveness by adhering to experimental design principles; and studies that align the intervention with the specific needs of students (e.g., decoding, vocabulary, and/or comprehension). We also acknowledge that essential aspects of reading comprehension with older students include consideration of engagement and involvement with text, motivation, self-efficacy, and how to nurture and expand reading interests. Many of these variables are considered to be primary sources of variance when attempting to positively influence the reading comprehension of older students with reading difficulties (Guthrie, Wigfield, & VonSecker, 2000). A better understanding of these key variables will assist teachers and educational decision makers in improving reading instruction for older students.

Biographies

MEAGHAN S. EDMONDS, PhD, is a research associate at the Vaughn Gross Center for Reading and Language Arts at the University of Texas at Austin, Meadows Center for Preventing Educational Risk, College of Education SZB 228, 1 University Station D4900, Austin, TX 78712-0365; msedmonds@mail.utexas.edu. She holds a doctorate in educational psychology, a master’s degree in curriculum and instruction, and an MEd in program evaluation. Her current research is focused on reading comprehension and policy evaluation.

SHARON VAUGHN, PhD, holds the H. E. Hartfelder/Southland Corporation Regents Chair in Human Development and is the executive director of the Meadows Center for Preventing Educational Risk at the University of Texas at Austin, College of Education SZB 228, 1 University Station D4900, Austin, TX 78712-0365; srvaughnum@aol.com. She was the editor in chief of the Journal of Learning Disabilities and the coeditor of Learning Disabilities Research and Practice. She is the recipient of the American Educational Research Association Special Education SIG Distinguished Researcher award. She is currently the principal investigator or coprincipal investigator on several Institute for Education Science, National Institute for Child Health and Human Development, and Office of Special Education Programs research grants investigating effective interventions for students with reading difficulties as well as students who are English language learners.

JADE WEXLER, PhD, is a research associate at the Meadows Center for Preventing Educational Risk at the University of Texas at Austin, College of Education SZB 228, 1 University Station D4900, Austin, TX 78712-0365; jwexler@mail.utexas.edu. Her research interests are interventions for adolescents with reading difficulties, response to intervention, and teacher education.

COLLEEN REUTEBUCH, PhD, serves as a project coordinator for the Center for Research of the Educational Achievement and Teaching of English Language Learners (CREATE) Project at the Meadows Center for Preventing Educational Risk at the University of Texas at Austin, College of Education SZB 228, 1 University Station D4900, Austin, TX 78712-0365; ckreutebuch@mail.utexas.edu. Her research interests include reading and content area interventions.

AMORY CABLE, PhD, is a speech-language pathologist and is currently writing summaries of research for a speech-language clinical database. Via Siciliani 44, Bisceglie, BA CAP 70052; amory.cable@gmail.com. KATHRYN KLINGLER TACKETT, MEd, is a doctoral candidate at the University of Texas at Austin in the Department of Special Education (in learning disabilities and behavior disorders), an assistant instructor with the Department of Special Education, and a research assistant with the Center on Instruction, Special Education Strand, which is housed at the Vaughn Gross Center for Reading and Language Arts, University of Texas at Austin, College of Education SZB 228, 1 University Station D4900, Austin, TX 78712-0365; katieklingler@mail.utexas.edu.

JENNIFER WICK SCHNAKENBERG is the associate director of the Texas Reading First Initiative at the Vaughn Gross Center for Reading and Language Arts at the University of Texas at Austin, College of Education SZB 228, 1 University Station D4900, Austin, TX 78712-0365; jennwick@mail.utexas.edu. She provides technical assistance to state-level, district-level, and campus-level personnel. She trains personnel on using assessment, implementing the three-tier model effectively, and providing effective and comprehensive reading instruction to all students. In addition, she supervises technical assistance and professional development team members.