Effects of Classroom Practices on Reading Comprehension, Engagement, and Motivations for Adolescents

Abstract

We investigated the roles of classroom supports for multiple motivations and engagement in students’ informational text comprehension, motivation, and engagement. A composite of classroom contextual variables consisting of instructional support for choice, importance, collaboration, and competence, accompanied by cognitive scaffolding for informational text comprehension, was provided in four-week instructional units for 615 grade 7 students. These classroom motivational-engagement supports were implemented within integrated literacy/history instruction in the Concept-Oriented Reading Instruction (CORI) framework. CORI increased informational text comprehension compared with traditional instruction (TI) in a switching replications experimental design. Students’ perceptions of the motivational-engagement supports were associated with increases in students’ intrinsic motivation, value, perceived competence, and increased positive engagement (dedication) more markedly in CORI than in TI, according to multiple regression analyses. Results extended the evidence for the effectiveness of CORI to literacy/history subject matter and informational text comprehension among middle school students. The experimental effects in classroom contexts confirmed effects from task-specific, situated experimental studies in the literature.

The purpose of this study was to examine the extent that language arts instruction embedded with explicit supports for student engagement and motivation in the form of Concept-Oriented Reading Instruction (CORI) increases information text comprehension and student motivation, in comparison with traditional instruction (TI). Distinctly from previous research on CORI, which investigated literacy integrated into science, this study examined integrated literacy–history instruction. Also, it builds on prior studies by examining CORI in the middle school context rather than the elementary school context. Moreover, this study uniquely used a within-subjects experimental design that more rigorously controls individual differences of students, teachers, and classrooms than previously employed quasi-experimental designs. We also examined whether the students’ perceptions of motivation-supporting practices were associated with increases in multiple constructs of motivation and engagement in CORI and TI.

Literature Review and Theoretical Framework

Description of CORI

The central goal of the current version of CORI was to provide language arts instruction integrated with history content, emphasizing the Civil War for middle school students. We aimed to assure that students were engaged in reading informational text in class and for homework. Engaged reading with informational text refers to active text interaction in which students are seeking conceptual understanding of complex topics. In this pursuit, students are energized by internal motivations such as intrinsic motivation, value, and perceived competence. They believe that reading extensively and deeply is beneficial to them in the immediate present. Engaged readers are strategic, using cognitive tools such as concept mapping to organize text-based knowledge, and they socially share the knowledge construction process and products with classmates and other audiences. CORI was intended to enable students to be engaged in reading, text-based writing, and discussion for at least 75 minutes in a 90-minute reading/language arts period. Materials were mainly informational texts; as defined by Duke (2000), these texts contain descriptions of the natural and social worlds that are presented as factual. Biographies were used extensively. Intensive teacher planning was needed to empower students into this academic literacy.

In this middle school CORI implementation, four motivational-engagement supports were provided by teachers: (1) competence support (to assure perceived competence), (2) providing choice (to increase intrinsic motivation), (3) emphasizing importance of reading (to foster value), and (4) arranging collaboration (to increase amount of engaged reading activity). The practice of competence support refers to teacher actions that build perceived competence by providing texts that are readable, providing feedback on accomplishment, recognizing students’ knowledge gained from reading, and helping students set realistic goals. Note that although we measured students’ perception of the practice, we did not train teachers to provide competence support explicitly because of their level of familiarity with it and our limitations of time and resources. The practice of providing choice refers to affording self-selection of books or sections of books, inviting student input into subtopics of study, providing options for demonstrating learning, and self-selecting partners for reading. The practice of emphasizing importance refers to affording concrete experiences of building knowledge from text, asking students to explain to peers how reading complements videos, or relating text ideas to personal background knowledge. The practice of arranging collaboration refers to enabling students to read as partners, exchange ideas from text, lead discussion groups, participate in group projects such as poster making, and exchange feedback with peers.

Previous Studies of CORI’s Effects on Achievement, Motivations, and Engagement

Prior CORI studies at the elementary level focused on the simultaneous implementation of the following practices: providing choices (of books, learning tasks, or ways of showing reading proficiency), arranging collaborations (partnerships, team projects), supporting competence (matching text difficulty to learner competencies, adjusting learning tasks to student needs), and offering relevance (hands-on science activities, authentic audiences for writing, personal persuasive essays). Emphasizing importance (helping students recognize text as their primary source of learning) was added for the middle school level (Guthrie, Klauda, & Ho, 2013). These motivational-engagement practices were combined with direct strategy instruction.

Previous evidence of CORI effectiveness was based on equivalent-groups quasi-experimental designs comparing students receiving CORI with other students who received TI or strategy instruction. Selected findings from a meta-analysis of 11 studies and 75 comparisons showed that CORI had effect sizes of the following: 0.91 for standardized reading comprehension, 0.73 for experimenter-constructed informational text comprehension, 1.20 for intrinsic motivation, 0.49 for self-efficacy, and 0.49 for amount of reading. In this study, we used a within-subjects design, termed switching replications, in which all students received both CORI and TI in counterbalanced order. This design is quite sensitive to the instructional treatment because all student and teacher variables that might be confounded with the instruction are necessarily controlled. For this study, our first expectation was that CORI would be associated with higher informational text comprehension achievement than TI would.

In prior research, influences of CORI on reading achievement, motivations, and engagement have been attributed to the composite of motivational practices combined with strategy instruction (Guthrie, McRae, & Klauda, 2007). In that work, little attempt was made to identify the extent that motivational and engagement supports are linked to increases in motivations apart from their combination with strategy instruction. In this investigation, we assessed students’ perceived instructional emphasis on four motivational practices in both CORI and TI. It is clear that teachers in TI employ practices to motivate students, and we expected to observe positive effects of these practices in both instructional contexts. However, because CORI provides specific guidance for teachers in using the motivational-support practices, we expected them to have more benefits for students’ motivation and engagement in CORI than similar practices in the TI framework which did not provide such guidance would. Thus, our second expectation was that students’ perceived level of motivational-engagement support would increase motivation and engagement more in CORI than in TI.

Theoretical Perspectives on Motivation and Engagement

Rationale for Multiple Motivational-Engagement Supports

CORI is essentially a pedagogy, consisting of sets of interleaved procedures for fostering students’ acquisition of academic literacy. To increase motivation in a brief classroom intervention, we drew on four theoretical frameworks to identify motivational-engagement supports. CORI incorporates those practices directed toward specific motivations consisting of the following: autonomy support for intrinsic motivation, emphasis on importance for valuing reading, success and goal setting for self-efficacy, and collaborative activities for social motivation. Their frameworks are briefly stated next.

Each of the following theories forward a central organizing motivational construct, with proposed environmental supports. First, self-determination theory proposes that intrinsic motivation, consisting of self-direction and positive affective involvement (Zhou, Ma, & Deci, 2009), generates behavioral engagement in a domain such as reading (Ryan & Deci, 2009). In this perspective, the authors recommend autonomy support in the classroom, referring to shared control between the teacher and students. When teachers encourage students to provide input into instruction, and link their interests to learning activities (Zhou et al., 2009), students’ commitments (motivation) and participation (engagement) are expected to increase, which enhances their achievement (Reeve, 2012).

Second, expectancy–value theory (Eccles & Wigfield, 2002) places the motivational construct of value at the center of its formulation. In reading, value is the perceived importance of reading, which may encompass utility benefits or intrinsic merits of the domain. An individual’s value for reading predicts academic success, in the form of course grades, and involvement, in the form of behavioral engagement in reading (Durik, Vida, & Eccles, 2006). Experimentally, investigators have shown the effects of describing how information in the text would provide immediate professional benefit for the readers. Groups receiving emphasis on the benefits of a text showed enhanced behavioral engagement (close attention to reading) and increased conceptual comprehension of informational text compared with no-rationale groups (Jang, 2008 ; Reeve, Jang, Hardre, & Omura, 2002). Furthermore, asking students to find personal usefulness and applicability in math activities and math class increased students’ performance in both laboratory and classroom settings compared with control conditions (Hulleman, Godes, Hendricks, & Harackiewicz, 2010).

Third, sociocognitive theory (Bandura, 2001) presents self-efficacy as a driving motivational construct that influences an individual’s aspirations, resilience, and self-regulated activity (Schunk & Mullen, 2012). Self-efficacy refers to expectations for success in the future and logically depends on perceived competence, which refers to judgment of one’s current capacity for reading well (Chapman & Tunmer, 1995). There is high similarity among self-efficacy, self-concept of ability (Kurtz-Costes, Ehrlich, McCall, & Loridant, 1995), and perceived competence, all of which correlate positively with reading proficiency and reading volume (Katzir, Lesaux, & Kim, 2009). One study showed that self-efficacy and perceived competence formed one psycho-metric factor in a variety of domains for high school students (Hughes & White, 2011). In this study, we employed the construct of perceived competence because of its advantages for unequivocal measurement among young adolescents, and we relate it to the literature on self-efficacy because of the conceptual similarity of the constructs. Studies have shown that providing competence support through feedback on progress and helping students set realistic goals in specific reading tasks increases self-efficacy for the academic domain of reading (Schunk & Zimmerman, 2007). Competence support may also appear in the form of using disciplinary texts, such as science trade books, that enable students to build text meaning related to their observations and knowledge of the world while learning reading skills (Guthrie et al., 2007).

The fourth conceptual framework is activity theory (Leontiev, 1981), which emphasizes shared cognitive activity. Also contributing to this framework are Scribner and Cole (1981), who focused on the acquisition of culturally significant literacy practices, based on Vygotsky’s (1978) view of collaborative scaffolding in the zone of proximal development. Extended to classrooms by Gutiérrez and Lee (2009), this perspective highlights the collaboration between teachers and students and among students that empowers students to acquire literacy practices and accompanying cognitive proficiencies (Scribner & Cole, 1981). In the absence of personal connections, students’ engagement declines, leading to disengagement (Griffiths, Lilles, Furlong, & Sidhwa, 2012) and dropping out of school (Rumberger, 2011). Thus, collaborations of teachers and students and among students were expected to influence students’ reading motivations and engagement.

Rationale for Affirming and Undermining Forms of Motivation and Engagement

Engagement in academic work such as reading has often been defined and measured in the positive form. Behavioral engagement at school has been described as time, effort, and persistence in academic activity (Fredricks, Blumenfeld, & Paris, 2004). This is akin to the view of Skinner, Kindermann, and Furrer (2009) that engagement is a fusion of behavior and positive affect. We use the term dedication to refer to the positive form of behavioral reading engagement. Dedication is the commitment of time, effort, and perseverance to reading activities. Because reading is inherently cognitive, some of the effort in behavioral engagement is cognitive. However, there is a distinction between behavioral and cognitive engagement. Cognitive engagement is intention and effort in performing critical feedback loops in self-regulated learning that are cognitive and metacognitive in nature (Cleary & Zimmerman, 2012). Dedication entails a commitment to doing the reading but not necessarily optimizing the cognitive or metacognitive effectiveness of the reading. Furthermore, dedication is not devoid of motivation but relies heavily on intrinsic motivation, perceived competence, and value, as shown in this study, as well as other motivations (Reeve, 2012).

Some researchers have also investigated engagement in negative forms of disengagement (Juvonen, Espinosa, & Knifsend, 2009) or disaffection (Skinner et al., 2009), which correlate negatively with achievement. Referring to attempts to evade academic work and minimize effort, avoidance has been shown to correlate negatively to reading achievement in elementary school (Baker & Wigfield, 1999) and middle school (Long, Monoi, Harper, Knoblauch, & Murphy, 2007). However, engagement and disengagement are not simple opposites, as they correlate at about −0.65 for middle school students, sharing about 40% of their variance. This moderate correlation is due to a substantial proportion of students who report low engagement (not actively participating in reading) and low avoidance (not evading necessary work).

Motivational constructs have also been studied in their affirming and undermining forms. For example, perceived difficulty of reading and self-efficacy both contributed unique variance to achievement in primary-age children, although their measures appear to be nearly polar opposites (Chapman & Tunmer, 1995). Likewise, devaluing has been associated with achievement independent of other motivations (Legault, Green-Demers, & Pelletier, 2006). Although devaluing appears to be the inverse of valuing, a moderate correlation between them has been observed (Guthrie et al., 2013). An inspection of scatterplots revealed that portions of students report both valuing and devaluing reading, possibly because of the diversity of texts and reading tasks in schools.

In light of their potentially semi-independent contributions, we investigated the extent that affirming and undermining motivations and both the positive form of engagement (dedication) and the negative form of engagement (avoidance) were associated with classroom motivational support.

Processes of Disciplinary Literacy

According to C. Shanahan, Shanahan, and Misischia (2011), central processes in disciplinary reading include contextualization, corroboration, sourcing, text structure, graphic elements, and critique, all of which occur in both history and chemistry. Reisman and Wineburg (2008) also emphasize contextualization, focused on perspective taking. In CORI, we used secondary sources, consisting of trade books and biographies, except for a very few iconic documents such as the Emancipation Proclamation. In previous versions of CORI using biology, the texts were similar, consisting of secondary trade books and textbooks, as well as student-generated inquiry notes and outcomes.

In both history and science, CORI placed a priority on the process of perceiving text structure, as represented in a hierarchical pyramid. This requires effortful learning but yields dividends in comprehension. History and science share this dominance of pyramid structures. Across the CORI unit, main ideas included economics, politics, leadership, culture, and military factors. Concept maps in pyramid shape were built from text for each main idea with respect to causes of the Civil War and a second set for outcomes of the war. Students filtered biographies into the structures to show individuals’ contributions.

A second process of disciplinary literacy that is highly relevant to our study is contextualization. To foster this process, students performed a culminating project in which each team adopted either the Northern or Southern perspective to describe the outcomes of the Civil War. This viewpoint determined students’ interpretations of costs and benefits of military, economic, political, and cultural outcomes. For example, from the Southern perspective, the economic losses in the South led to widespread poverty and continued political defiance. But from the Northern perspective, Southern economic losses represented new entrepreneurial opportunities during Reconstruction.

At this level, contextualization is quite similar in reading history and science. A text consisting of “the cell could not be observed because it was too dark” has different meanings depending on whether it is a leaf cell or a prison cell. The type of leaf or the location of prison also influences interpretation. These contextual effects are profoundly important to understanding a large majority of informational texts (Meyer et al., 2010) in science or history. Consequently, we believe that teaching these processes to seventh graders will be useful across subject matter areas. Such broad contextualization processes that are general to history and science can be measured and are correlated with diverse reading required in high school (Ozuru, Dempsey, & McNamara, 2009) and standardized tests in prior CORI studies (Guthrie et al., 2004). Consequently, it is the generalizable text structuring and contextualization processes widely needed in history and science reading that we incorporated into CORI.

Issues for This Investigation

This study extends prior CORI research by using history (U.S. Civil War) as the subject matter context, whereas all previous CORI studies by this team used science topics of ecology, symbiosis, adaptation, biodiversity, and endangered species (Guthrie et al., 2007, 2013). Also, this study examined middle school students, whereas all previous studies except Guthrie et al. (2013) included students in grades 3–5 only. Finally, this study uniquely investigates associations of student perceptions of teachers’ motivational support with changes in student motivation and engagement.

Questions for Investigation

1. To what extent does CORI that integrates history with literacy instruction enable students to acquire competencies in informational text comprehension in history more effectively than TI for reading/language arts for middle school students?

2. To what extent does the set of CORI practices of emphasizing support for competence, choice, value, and collaboration facilitate motivation and engagement in comparison with TI for middle school students?

Hypotheses

1. Students provided with CORI will show higher informational text comprehension in history than students provided TI.

2. Students’ perceived level of motivational-engagement support will increase motivation and engagement more in CORI than in TI.

Method

Participants

The participants were 615 grade 7 students attending four middle schools in a rural public school district of a mid-Atlantic state during the 2010–2011 school year. They were instructed by 11 reading/language arts teachers. Although all 1,200 seventh graders in the district completed the study measures, student data were not analyzed for seven reading/language arts teachers. Two teachers were excluded because one moved and a second taught CORI during both cycles of the study, to complete the CORI unit for the teacher who moved. In addition, three teachers entered the school in the middle of the academic year and could not be provided the two half-days of professional development, which was given only at the end of the preceding summer according to school policy, and two teachers were below the threshold in implementation, as described in the instructional fidelity section. Students taught by the remaining 11 teachers were additionally excluded if their parents withheld consent for their data to be used (26 students), patterns in the students’ responses to the study measures indicated that they did not follow instructions (15 students), or they were suspended or engaged in home-schooling during any portion of the study (four students). Finally, students were excluded if they had Individualized Education Plans (IEPs; 94 students) because they were provided with a wide variety of instructional interventions within and outside the classrooms that the researchers could not supervise or guide.

The final student sample was 47.0% male, 78.9% European American, 16.7% African American, 3.4% Asian American, and 1.0% other ethnicities or unknown. A total of 20.3% of students were eligible for free and reduced-price meals (FARMs). Less than 1% was receiving English as a second language instruction. The total grade 7 population in the school district was 50.4% male, 75.0% European American, 19.8% African American, 3.3% Asian American, and 1.9% other ethnicities, and 27.3% was receiving FARMs and less than 1% receiving English as a second language instruction. The studied sample had lower percentages of males, African Americans, and receiving FARMs than the full district sample did. We do not claim that the sample represented the school district but that the samples receiving the two instructional treatments did not differ in any demographic characteristic, which was assured by the switching replications design.

The 11 teachers who instructed the final sample were 10 European American females and one African American female. Their teaching experience ranged from one to 28 years, with a median of eight years. Their range of experience teaching at their current school was one to 22 years, with a median of five years. The highest degree earned by three teachers was a master’s; the rest held bachelor’s degrees.

Design

We used a switching replications experimental design to investigate the effects of a composite of motivational-engagement supports on informational text comprehension achievement (hypothesis 1). According to a widely used methodological text, “The Switching Replications design is one of the strongest of the experimental designs” (Trochim, 2006, para. 5). Each student participates in the treatment group and the control group. The implementation of the treatment is repeated or replicated. In the repetition of the treatment, the two groups switch roles; the original control group becomes the treatment group in phase 2, while the original treatment group acts as the control. In this study, each student’s increase in text comprehension is compared under the treatment and control conditions. All individual differences of preexisting ability, interest, attention, comprehension, or demographic factors are controlled precisely. Each person, with all of his or her idiosyncrasies, is given both the treatment (CORI) and control (TI) conditions. In this situation, individual differences cannot be confounded with the instructional condition. Note that it is unnecessary to use a between-subjects control (e.g., a covariate) in this procedure.

According to Shadish, Cook, and Campbell (2002), the switching replications design is strong because it “controls most threats to internal validity, and it enhances external and construct validity” (p. 192). Previously called the within-subjects experimental design (Winer, 1962), the switching replications design controls for preexisting individual differences more effectively than a randomized design does because students are compared with themselves. Switching replications designs have been used effectively in experimental studies of reading comprehension strategy instruction for seventh-grade students (Bulgren, Marquis, Lenz, Deshler, & Schumaker, 2011), instruction in basic reading processes (Huemer, Mikko, Landerl, & Lyytinen, 2010), an intervention to promote civic engagement (Dabelko-Schoeny, Anderson, & Spinks, 2010), and instruction for math achievement (Bottge, Rueda, LaRoque, Serlin, & Kwon, 2007).

The switching replications design is advantageous when teachers or administrators want all students to participate in the instructional treatment, which was true in our study. In this design, half of the students received CORI first, while others were the control group, and half received CORI second, while the original experimental group became the control. Not only are all students included, but schools also only need to allocate enough resources to give the program to half of the students at a time.

In this design, classes were randomly assigned to group 1, which received CORI first and TI second, or group 2, which received TI first and CORI second. All teachers taught the same students in both instructional approaches, and all students received both instructional treatments. The data analysis to examine the effects of instruction on text comprehension was a repeated measures ANOVA.

Measures

Reading Motivation and Engagement

We constructed a self-report survey that inquired about students’ reading motivations, defined as beliefs, values, and goals related to reading, and their reading engagement, defined as time, effort, and persistence in reading activities (Guthrie, Wigfield, & You, 2012) in their reading/language arts class for the previous four weeks. We wrote the 42 items in the measure, or six items for each of seven constructs. Each item had four response options: “not at all true of me,” “not very true of me,” “somewhat true of me,” and “very true of me.” Three of the seven constructs represent affirming motivations for reading, or motivations typically associated positively with achievement, whereas two constructs represent undermining motivations for reading, or motivations typically associated negatively with achievement (Legault et al., 2006).

The conceptual bases for the constructs are drawn from multiple theoretical perspectives. First, among the affirming motivations, intrinsic motivation reflects interest and enjoyment in reading and the desire to read often (Gottfried, Fleming, & Gottfried, 2001). The second affirming construct was value, or the belief that reading is important and useful for one’s future (Trautwein, Lüdtke, Schnyder, & Niggli, 2006). The third affirming construct, perceived competence, is belief in one ’s capacity to complete reading tasks successfully (Schunk, 2003). One undermining construct, devalue, refers to the belief that reading is not important or useful for one ’s success or future (Legault et al., 2006); the other, perceived difficulty, is the perception that reading tasks are difficult (Chapman & Tunmer, 1995). The undermining constructs were used for descriptive correlational purposes and were not used to test the hypotheses. For purposes of parsimony in the statistical analyses, the affirming constructs, which correlated moderately positively, were used in a composite to test hypothesis 2.

The final two constructs represented behavioral forms of engagement (Fredricks et al., 2004) in reading: the positive construct of dedication, which emphasizes persistence, effort, time expenditure, and intention to learn from reading (Guthrie et al., 2012), and the negative construct of avoidance, which emphasizes having an aversion toward reading and, therefore, minimizing the time and effort expended on it (Meece & Miller, 2001). All items for each engagement and motivational construct are presented in Appendix A.

Exploratory factor analyses were conducted to finalize the motivation subscales, using data from the April (second) administration of the measure. We conducted principal components analyses with varimax rotation separately for each construct. The number of eigenvalues greater than 1, scree plot, and variance explained, and factor loadings ≥ 0.40 were evaluated when constructing scales. The Cronbach’s α values for each scale in March, April, and June, respectively, were: intrinsic motivation = .88, .72, and .88; value = .78, .79, and .79; perceived competence = .75, .78, and .82; devalue = .79, .81, and .81; and perceived difficulty = .80, .81, and .81. Each of these scales correlated with two measures of engagement, dedication and avoidance, and informational text comprehension, showing predictive validity (Guthrie el. al, 2013). Cronbach’s α values for the engagement scales in March, April, and June, respectively, were as follows: dedication = .79, .81, and .82; and avoidance = .84, .85, and .88. In a prior study, both dedication and avoidance were significantly correlated with informational text comprehension prior to intervention, showing predictive validity (Guthrie et al., 2013).

Perceptions of Instruction

We constructed a self-report questionnaire representing four constructs to assess students’ perceptions of their reading/language arts teachers’ classroom practices and supportive behaviors during the previous four weeks of instruction. First, importance support consisted of teacher emphasis on the value and utility of reading and writing as expressed in the expectancy–value theory (Eccles & Wigfield, 2002). Choice support concerned the provision of significant opportunities for autonomy and self-direction in reading (Zhou et al., 2009). Collaboration support comprised the fostering of cooperation and interpersonal relations both between students and their teachers and among students within the context of reading and writing (Wentzel, 2010). The fourth construct, competence support, was defined as encouraging students to succeed in reading by providing instrumental help and verbal reinforcement. Competence support was measured with 12 items based on previous measures that assessed student perceptions of supportive help, expectations, feedback from teachers, and instrumental help (Skinner & Belmont, 1993). The response options for all perceptions of instruction items were “strongly disagree,” “slightly disagree,” “slightly agree,” and “strongly agree.” Cronbach’s α values for the scales were as follows in March, April, and June, respectively: importance support = .74, .82, and .84; choice support = .67, .75, and .77; collaboration support = .72, .80, and .81; and competence support = .74, .76, and .81.

Reading Fluency

The Woodcock–Johnson–III Reading Fluency Test (Form C in March, Form B in April, and Form A in June) was employed. This test gives students three minutes to silently read as many sentences as they can, indicating whether each one is true or false. In other words, this assessment evaluates speed of reading and literal understanding at the individual sentence level. Raw scores (number correct minus number incorrect) were converted to standardized scores for use in analyses. The measure showed test–retest reliability correlations of 0.83 for both March–April and April–June.

Informational Text Comprehension

The overall ability to comprehend informational text was assessed with a researcher-developed measure, reflecting a view of informational text comprehension as an interactive process that leads to literal understanding, as well as formation of a structured network of knowledge representing the information in the text, fused with the reader’s prior knowledge and experiences related to the text topic. The assessment consisted of two 60–100-word and three 250–300-word passages on historical topics that differed from the topics of instruction. The shorter passages were at fifth- to seventh-grade reading levels, whereas the longer passages were at eighth-grade to college reading levels. Each passage was followed by five multiple-choice items, for a total of 25 items. The items for the shorter passages required exact or near paraphrases of text, which, at most, involved linking information in two consecutive sentences. There were four item subtypes: literal understanding, word in context, phrase understanding, and basic conceptual understanding. The items for the longer passages required more complex reasoning and text analysis, involving integration of two or more consecutive or nonconsecutive text propositions with each other and/or background knowledge. There were five item subtypes for these passages: main concept, sub-concept, relational understanding, partial passage synopsis, and full passage synopsis (see Appendix B).

Three alternate test forms were employed, with one common or linking passage across forms and four unique passages per form. The forms were counterbalanced so students received different forms at each assessment point, and approximately equal numbers of students received each form. Due to low item–total correlations (<.10), four items (one linking passage item, two unique passage items from Form A, and one unique passage item from Form C) were deleted before raw scores were calculated. Raw scores on the three forms were equated for difficulty at each timepoint through linear equating (Crocker & Algina, 1986). In accord with Livingston (2004), linear equating rather than item response theory equating was employed, as the assumptions of linear equating were more realistic, and the benefit of flexibility offered by item response theory (e.g., for creating adaptive tests) was not relevant for the present study. Because the item deletions resulted in uneven numbers of items per form, the raw scores were calculated and equated in the form of percentage correct values (Livingston, 2004). As there were only four linking items that were not completely representative of the rest of the test items, the linking items were excluded in the initial calculation and equating of raw scores.

The final percentage correct scores were calculated by weighting raw percentage correct scores on the linking items by 4 (the number of linking items) and weighting equated percentage correct scores on the unique passage items by the number of those items on a given form (i.e., 18 for Form A, 20 for Form B, 19 for Form C). Cronbach’s α values for each form were as follows in March, April, and June, respectively: Form A = .70, .75, and .73; Form B = .73, .76, and .76; and Form C = .74, .75, and .79. Although this was the first study with this scale, a highly similar scale in science text comprehension had significant correlations with grades in reading/language arts and fluency, showing predictive validity (Ho & Guthrie, 2013).

Procedures for Data Collection

The assessment battery was administered to students in their usual reading/language arts classrooms by their teachers, with oversight and assistance from project personnel, three times during the school year: the week prior to the first CORI intervention (March), the week after the first four-week CORI intervention concluded and before the second CORI intervention commenced (April), and the week after the second CORI intervention concluded (June). Testing occurred during students’ two daily 45-minute reading/language arts periods on one day at each assessment point. Project personnel administered the tests to absent students in small groups on the day following each primary assessment, if possible. For each assessment, the test administrators read aloud directions and reviewed sample items, then students completed each assessment silently, with strict time limits for all but the motivation assessments. The assessments were administered in the following order: information text comprehension (22-minute time limit), perceptions of instruction (5–10 minutes), motivation/engagement (7–10 minutes), Woodcock–Johnson–III Reading Fluency Test (3-minute limit), and inferencing (12-minute limit). Demographic data were obtained from the central office of the school district.

Instructional Conditions

CORI

A total unit framework for CORI on the topic “U.S. Civil War: Causes and Outcomes” is provided in Appendix C. Across the four-week period, it displays ingredients of motivational-engagement support, strategy instruction, content concepts, and the central text for instruction of on-grade, struggling, and advanced readers. Also displayed is a weekly framework, showing the activities and instruction for week 1.

For this CORI unit, there is a knowledge superstructure in pyramid form. At the apex is the main idea, the causes and outcomes of the Civil War. Subordinated to it are key concepts, consisting of culture, economics, leadership, politics, beliefs, military, and slavery. With these concepts and evidence/examples for each, students read to explain first the causes and then the outcomes of the war. For example, the victory and/or defeat in a given battle can be explained with this concept structure. Contributions of visible characters, such as Harriet Tubman, can be represented by using these integrative concepts. For example, in week 1, days 1–3, the daily driving questions for instruction were “What were the similarities and differences in the economies of the North and the South?” (Monday); “What were the key aspects of Northern and Southern cultures?” (Tuesday); and “How do you explain the different views about slavery in the North and the South?” (Wednesday). In the sense that full text comprehension entailed perceiving the causes and outcomes of the Civil War from either the Northern or Southern perspective, the students’ reading may be characterized as disciplinary literacy (T. Shanahan & Shanahan, 2008). However, the assessment of informational text comprehension did not require a high level of perspective taking, although it demanded multitiered knowledge structuring, which is also needed for fully comprehending middle school science texts.

In this middle school CORI implementation, four motivational-engagement supports were provided by teachers: competence support (to assure perceived competence and reduce perceived difficulty), providing choice (to increase intrinsic motivation), emphasizing importance of reading (to foster value and decrease devaluing), and arranging collaboration (to increase each motivation). Practices were expected to increase dedication (positive engagement) and reduce avoidance (disengagement).

The practice of competence support refers to teacher actions that build perceived competence by providing texts that are readable, providing feedback on accomplishment, recognizing students’ knowledge gained from reading, and helping students set realistic goals. Note that although we measured students’ perception of the practice, we did not train teachers to provide competence support explicitly because of their level of familiarity with it and our limitations of time and resources. The practice of providing choice refers to affording self-selection of books or sections of books, inviting student input into subtopics of study, providing options for demonstrating learning, and self-selecting partners for reading. The practice of emphasizing importance refers to affording concrete experiences of building knowledge from text, asking students to explain to peers how reading complements videos, or relating text ideas to personal background. The practice of arranging collaboration refers to enabling students to read as partners, exchange ideas from text, lead discussion groups, participate in group projects such as poster making, and exchange feedback with peers. The rationale for placing the motivational-engagement supports in weekly order was that teachers could focus and implement them more readily than if we expected teachers to implement them simultaneously. Teachers scaffolded each type of motivational-engagement support as they would cognitive support.

Within CORI for middle school history, teachers provided cognitive scaffolding in the forms of strategy instruction for inferencing, summarizing, and concept mapping. We selected inferencing because informational texts are often densely packed forms of discourse, and students cannot easily link background knowledge to them. We selected summarizing because students must build hierarchies of knowledge on a firm basis of accurate text representation. Finally, we selected concept mapping because it enables students to restructure text-based and prior knowledge and imbue them with causal and temporal linkages. Each cognitive strategy was taught with direct instruction consisting of teacher modeling, scaffolding, and guided practice in the strategy with a variety of texts on the conceptual theme. Students wrote inferences, summaries, and concept maps. They collected them with project notes in individual portfolios and displayed them in their culminating posters. We sequenced these tasks from simple to complex and based them on smaller to larger amounts of text. This assured initial success and ultimate complexity in students’ cognitive processing of new informational texts.

Books on the Civil War unit consisted of expository trade books on policies, battles, economic issues, and leaders, as well as biographies of individuals and portraits of groups such as African American soldiers and women. These ranged in length and difficulty from 40 pages at the grades 3–5 levels for struggling readers to 50–70 pages at the grades 6–8 levels for on-grade students. We compiled a set of articles from history journals at grade 9 to college levels in 70 pages of 8.5″ × 11″ sheets in a smaller font for the honors students. Each class had three titles as class sets, 10 titles as group sets, and 19 individual books for independent reading.

Videos of Civil War topics were drawn from the Discovery Channel’s online service, United Streaming, and were provided to teachers on flash drives. Segments of four to eight minutes were used to stimulate interest and afford background knowledge. In weeks 1–4, respectively, the frequencies of the videos were reduced from five to three to two to none. The real-world connections provided by the videos were no longer needed by week 4.

Daily instruction typically began with 15 minutes of whole-class instruction centered on the knowledge goals of the week and day. Teachers helped students unpack complex guiding questions and relate them to prior work. For each cognitive strategy, comprehension instruction began with whole-class modeling of cognitive strategies, initially lasting 20 minutes and gradually decreasing to no time by the end of the week. At the beginning of each class, motivational-engagement support was organized and explicitly announced. For example, during the emphasis on collaboration support, teachers communicated, “You will be working as partners today,” or “You will work with your team, and you should pay attention to working well with your classmates.” Approximately 15 minutes of guided reading was provided for struggling readers, on-grade readers, and advanced readers, in that order. When they were not in guided reading groups, students either performed text-based writing or independent reading. Students were placed into flexible groups by the teacher in consultation with the instructional resources teacher in each school.

TI

TI was provided by the same teachers to students in reading/language arts classes during the control period for each class. This was “instruction as usual” in the school. Teaching resembled the Directed Reading–Thinking Activity framework (Stauffer & Harrell, 1975). Goals of this instruction for literary text comprehension included understanding character development, plot, symbolism, and themes. A well-known anthology of literature was used, which included informational texts such as description of historical settings for stories and characters. Typically, students discussed themes from previously read sections of text, read new sections, and interacted socially to represent them as accurately as possible. Teachers provided cognitive scaffolding for analysis of episodes and integration of salient texts. Students shared their opinions about characters’ actions and traits. Informational text comprehension was taught by scaffolding students in the cognitive skills of analyzing and summarizing letters, documents, and historical background to explicate literary texts.

Instructional goals for informational text reading included the following (see Appendix D):

• Apply comprehension skills by selecting, reading, and interpreting a variety of print and nonprint informational texts, including electronic media.

• Analyze important ideas and messages in informational text.

• Read critically to evaluate informational text.

• Read orally at an appropriate rate.

• Read grade-level text with both high accuracy and appropriate pacing, intonation, and expression.

• Develop and apply vocabulary through exposure to a variety of texts.

• Apply a conceptual understanding to new words.

• Understand, acquire, and use new vocabulary.

• Apply comprehension skills through exposure to a variety of print and nonprint texts, including traditional print and electronic texts.

Goals were placed on the blackboard or whiteboard daily. These goals for informational text reading were shared in both TI and CORI.

During class, TI teachers typically began with a review of the previous two or three lessons and provided an overview of the literary or informational text for the day. Students then volunteered to read several text pages aloud individually. Teachers asked focused questions on the particulars of plot, character in literature, or concepts with evidence in the knowledge structure. About 70% of the questions could be answered with close text inspection, whereas 20% were open or conjectural, leading to free-ranging discussion. Issues of theme were raised in 2–3% of the questions, and direct reading strategy instruction consumed about 2–3% of the teacher–student interactions. During about 25% of the classes, writing was emphasized, including personal narrative, exposition, and text summaries. To support writing, two or three genre-specific goals were discussed, posted, and graded for each type of composition. Monthly writing grades were given.

Assessments of class progress were given, consisting of three to five short constructed-response questions weekly. Teachers graded about 50% of these on three-level rubrics of “outstanding,” “acceptable,” and “needs improvement.” Homework was given daily, consisting of reading several pages in the anthology, and posted on the board. Students read about 10 pages per week, which enabled them to finish the anthology of about 400 pages and also read Internet sources of about 20–25 pages per year. Grades were based on written assessments, compositions, class participation, and class attendance.

Teachers were monitored and coached by the instructional reading leader and the principal. They made announced classroom observations and held personal conversations with a few teachers while attending weekly team planning meetings occasionally. Meetings were often data based, with attention to class progress and especially performance of low-achieving or special education students. Special education services were provided in the form of part-time assistants within and outside the classrooms for the large majority of students with IEPs. About 2% of students with IEPs were taught reading/language arts in a targeted program in a different location.

TI did not include guidance for explicit motivational-engagement supports, although some teachers occasionally afforded opportunities for choice of texts and collaborative discussion. In both CORI and TI, informational texts and cognitive scaffolding for them were provided. Although motivational-engagement support was not denied to students in TI, it was less emphasized by most teachers, as shown in the students’ perceptions of instruction described in the Instructional Fidelity section.

Professional Development

Professional development in CORI was provided for two half-days to all teachers. We began with a CORI minilesson, in which teachers experienced the same motivational-engagement supports and strategy instruction that CORI provides to students. Two staff members worked with all teachers and instructional leaders to provide professional development in affording motivational-engagement support, implementing strategy instruction, selecting books for diverse students, and managing groups. A teacher’s guide was provided, which teachers adapted slightly to meet classroom constraints. Use of portfolios, rubric-based feedback, and portfolio grading were discussed.

Instructional Fidelity

One indicator of the extent to which the teachers implemented the two instructional conditions according to the designs for motivational-engagement support was students’ perceptions of instruction. As described previously in this Method section, students completed pre- and postinstructional questionnaires for both CORI and TI. We conducted a repeated measures ANCOVA with the students’ perceptions of each motivational-engagement support as the dependent variable and the order of instruction (CORI first vs. TI first) as the independent variable, along with the covariates of gender (1 = male; 2 = female), income (0 = did not receive FARMs; 1 = received FARMs), and reading fluency. These covariates were used due to their known association with reading comprehension (National Center for Education Statistics, 2011).

Although the analysis necessarily controls for levels of the covariates, we attempted to determine whether the covariates interacted with the independent variable. We did not compute a multivariate ANOVA because we were theoretically interested in each of the three instructional practices. However, as we tested the three variables (choice, importance, and collaboration support) that we expected to influence informational text comprehension, we applied the Bonferroni correction for multiple tests and accepted results as significant when p < .02. Although we conducted an ANCOVA with competence support as the independent variable, this was a separate analysis as a discriminant validity check, as we did not include this in the design for CORI, and consequently was not included in the count of tests for the Bonferroni correction.

With the students’ perceptions of the instructional practice of affording choice as the dependent variable, the ANCOVA showed no main multivariate effect of time. There was a significant interaction of time and order of instruction, F(2, 539) = 12.84, p < .001, as shown in Figure 1a. The other effects were not statistically significant. For students receiving CORI first, perceived choice support increased substantially in the first period and decreased slightly in the second period. For students receiving TI first, perceived choice decreased slightly in the first period and increased markedly in the second period. This shows that perceived choice support increased substantially during CORI and that perceived choice support decreased slightly during TI. This occurred for students regardless of whether students received CORI or TI first.

FIGURE 1.

Student Perceptions of Four Instructional Support Practices in Concept-Oriented Reading Instruction (CORI) and Traditional Instruction (TI) at Three Timepoints: (a) for Choice, (b) for Importance, (c) for Collaboration, and (d) for Competence

Note. Solid line = CORI first; dashed line = TI first.

To examine students’ perceptions of the instructional practice of emphasizing importance as the dependent variable, the ANCOVA showed no main multivariate effect of time. There was a significant interaction of time and order of instruction, F(2, 540) = 7.59, p < .001, as shown in Figure 1b. The other effects were not significant. For students receiving CORI first, perceived emphasis on importance increased slightly in the first period and decreased markedly in the second period. For students receiving TI first, perceived emphasis on importance decreased markedly in the first period and increased slightly in the second period. This shows that support for perceived importance increased slightly during CORI and that perceived importance decreased substantially during TI. This occurred for students in both orders. There was a significant interaction of order, time, and gender, F(2, 540) = 3.05, p < .05, in which females showed the pattern of the full group, and males with TI first showed the same pattern, while males receiving CORI first declined in importance perception in both instructional conditions. The other effects were not statistically significant.

With students’ perceptions of the instructional practice of arranging collaboration as the dependent variable, the ANCOVA showed no main multivariate effect of time. There was a significant interaction of time and order of instruction, F(2, 545) = 35.32, p < .001, as shown in Figure 1c. For students receiving CORI first, perceived collaboration support increased substantially in the first period and decreased markedly in the second period. For students receiving TI first, perceived collaboration support decreased slightly in the first period and increased markedly in the second period. This shows that perceived collaboration support increased substantially during CORI and that perceived collaboration support decreased noticeably during TI. This occurred for students in both orders. There was also a significant interaction of gender and order of instruction, F(2, 545) = 4.07, p < .02, indicating that females gained more than males from time 1 to time 2, but the two genders gained equally from time 2 to time 3. No other effects were statistically significant. For each instructional practice, the effect was p < .02, showing statistical significance for these multiple ANOVAs.

For the instructional practice of providing competence support as the dependent variable, the ANCOVA showed no main multivariate effect of time. There was no significant interaction of time and order of instruction (see Figure 1d). This is a form of discriminant validity for the instructional conditions because we did not provide any professional development for competence support, as we did for each of the other practices. Likewise, we did not provide any explicit information regarding competence support in the CORI teacher’s guide. Therefore, we did not expect an interaction between order and perceived competence support.

Fidelity of treatment was confirmed for the three aspects of engagement support that were trained and measured: perceived importance, collaboration, and choice. However, the aspect of engagement support that was not trained showed no association with instructional condition. This equal distribution of competence support across the two instructional conditions does not preclude the possible effects of competence support on student outcomes, which will be presented later.

Instructional fidelity was also identified through experimenter observation of each teacher for two 90-minute lessons. The observer used a scale of 1–3 (3 = exceptional; 2 = acceptable; 1 = needs improvement) to rate teachers on a rubric consisting of four categories: motivational-engagement support, guided practice, teacher feedback, and reading engagement. These dimensions had a median correlation of r = .63, p < .01, showing internal consistency among CORI attributes. Scores for each category were also summed, with all teachers in the analyses surpassing the threshold of 8 or higher out of 12 possible. Two teachers with scores below this threshold were eliminated. The motivational-engagement support dimension of the rubric correlated significantly with the sum of the students’ reports of perceived support for importance, collaboration, and choice, r = .51, p < .04, showing concurrence between observational ratings and students’ reports of teachers’ motivational-engagement support. Furthermore, students’ perceived motivational-engagement support correlated with the richness and completeness of their portfolios rated on the same scale, r = .65, p < .006.

Results

The first hypothesis was that the composite of engagement practices based on the framework of CORI will increase informational text comprehension in comparison with TI for reading/language arts. The statistical analysis was a repeated measures ANOVA. The dependent variables were the informational text comprehension measures given at three timepoints (Time: 1, 2, 3), and the independent variable was the Order factor that represented the sequence of instruction for each student (Order: CORI-TI or TI-CORI). Covariates were gender and income measured as FARMs and fluency measured by the Woodcock-Johnson–III Reading Fluency Test. Means, standard deviations, and correlations are shown in Tables 1 and 2, respectively.

TABLE 1.

Means (Ms) and Standard Deviations (SDs) of Informational Text Comprehension and Fluency by Gender, Income, and Instructional Group

	CORI cycle			Gender			Income
	Concept-Oriented Reading Instruction first	Traditional instruction first	Total	Male	Female	Total	No FARMs	FARMs	Total
Informational text comprehension
M	55.06	51.19	53.04	53.48	52.67	53.04	55.26	43.87	53.04
SD	15.88	14.24	15.16	14.14	15.79	15.16	14.33	15.11	15.16
N	283	308	591	274	317	591	476	115	591
Fluency
M	115.47	111.00	113.14	109.63	116.16	113.14	114.80	106.29	113.14
SD	15.41	14.86	15.28	13.23	16.26	15.28	14.82	15.31	15.28
N	282	307	589	272	317	589	474	115	589

Note. FARMs = free and reduced-price meals. This table contains data from measures administered and records of demographics at time 1.

TABLE 2.

Correlations Among Variables Used in the Switching Replications Analysis

	Gender	FARMs	Fluency	ITC	CORI cycle
Gender	—	.03	.21	−.03	−.06
FARMs	.03	—	−.22**	−.30**	.08*
Fluency	.21**	−.22**	—	.58**	−.15**
ITC	−.03	−.30**	.58**	—	−.13**
CORI cycle	−.06	.08*	−.15**	−.13**	—

Note. CORI = Concept-Oriented Reading Instruction; FARMs = free and reduced-price meals; ITC = informational text comprehension. Gender was coded as 1 = male, and 2 = female. FARMs was coded as 0 = pays for meals, and 1 = receives free or reduced-price meals.

^*p< .05.

^**p < .01.

The results were that none of the multivariate tests were statistically significant, except the Time by Order interaction, which was significant, F(2, 551) = 5.20, p < .006. More specifically, the linear effect for the Time by Order interaction was not statistically significant, but as expected, the quadratic effect was significant, F(1, 551) = 9.25, p < .002. In the between-subjects tests, Gender was significant, F(1, 552) = 7.21, p < .001; Income was significant, F(1, 552) = 32.50, p < .001; and Order was not statistically significant. Means and standard deviations are provided in Table 3. The effect size of the instructional advantage of CORI in comparison with TI was .26. This was a Cohen’s d computed from the F-value and the sample sizes (Thalheimer & Cook, 2002), which merges the effects at times 2 and 3.

TABLE 3.

Means and Standard Deviations of Informational Text Comprehension at Three Timepoints for Concept-Oriented Reading Instruction (CORI) and Traditional Instruction (TI) Groups

Statistic	Time 1		Time 2		Time 3
	CORI first	TI first	CORI first	TI first	CORI first	TI first
Mean	54.06	53.19	56.94	53.60	55.64	56.19
Standard deviation	15.41	14.10	16.70	17.55	17.09	17.02
N	269	288	269	288	269	288
95% confidence interval	[52.62, 55.50]	[51.80, 54.58]	[55.00, 58.67]	[51.92, 55.27]	[53.85, 57.42]	[54.46, 57.91]

The significant quadratic effect of the Time by Order interaction confirms the first hypothesis and is shown in Figure 2. From time 1 to time 2, the CORI first group (who received CORI then) increased, whereas the control group of TI first (who did not receive CORI then) did not change. From time 2 to time 3, the TI first group increased, but the CORI first group declined slightly. Thus, whenever CORI was taught, the group receiving it gained in informational text comprehension, and when TI was taught, the students did not change in comprehension level.

FIGURE 2.

Effects of Concept-Oriented Reading Instruction (CORI) and Traditional Instruction (TI) on Informational Text Comprehension (ITC) at Three Timepoints

Note. Solid line = CORI first; dashed line = TI first.

In the statistical analysis, time 1 was the preassessment for all students and shows comparable scores of 54.06% correct for the CORI first group (order 1) and 53.19% correct for the TI first group (order 2; see Table 3). The difference of the means was below the criterion of three standard errors. At time 2, the CORI first group, which had received CORI from time 1 to time 2, increased to a mean of 56.94, whereas the TI first group, which had received TI then, increased marginally to 53.60. A post hoc test showed that the difference of the groups at time 2 was statistically significant (p < .05).

At time 3, the TI first group, which had just finished CORI, increased to 56.19, which was a substantial upward increment. Post hoc tests showed this mean was significantly higher than the group’s mean at the outset in time 1 (p < .05) and was marginally statistically significantly higher than the CORI mean at time 3 (p < .10). Also at time 3, the CORI first group, which had just received TI, decreased to 55.64, which was a nonsignificant drop from their previous score of 56.94. This shows that the CORI first group retained most of the informational text comprehension competence gained in the original CORI intervention period.

It can be observed in Figure 2 that the group receiving CORI first (order 1) increased in the first instructional period, and the group receiving TI first (order 2) increased in the second instructional period. According to post hoc tests, the two instructional groups of CORI first and TI first had similar preassessment means in time 1 and similar postassessment means in time 3. Note that if the CORI students, who increased in the first period, continue to be engaged in the second period during TI, it would add to the apparent benefit of TI. The relative decrease for the CORI first group from time 2 to time 3 indicates that the impact on informational text comprehension is quite specific to the instructional conditions. We suggest that CORI increased informational text comprehension in comparison with TI when each student received both forms of instruction from the same teacher in the same classroom in two different instructional periods.

The effect of CORI on reading achievement was further investigated by conducting a nearly identical ANCOVA with fluency as the dependent variable. Instructional Order was the independent variable, with gender and FARMs as covariates. There was significant Order by Time interaction with a significant quadratic effect, F(1, 550) = 4.28, p < .03. The pattern was that when CORI was first, fluency reduced substantially in CORI and did not change in TI. When CORI was second, fluency did not change for either CORI or TI. Because fluency declined in CORI if it was provided first but not if it was provided second, we inferred that effects of CORI on fluency were inconclusive.

Prior to testing hypothesis 2, correlations among motivations, engagements, and instructional practices were examined (see Tables 4 and 5). As displayed for time 3 in Table 5, there was a distinct pattern of associations. Intrinsic motivation correlated positively with perceived competence support at .38 (p < .01); negatively with perceived difficulty, as expected, at −.20 (p < .01); positively with value at .51 (p < .01); negatively with devalue at −.67 (p < .01); negatively with avoidance at −.68 (p < .01); and positively with dedication at .57 (p < .01).

TABLE 4.

Correlations, Means, and Standard Deviations of Motivations and Engagement Practices at Time 1

	1	2	3	4	5	6	7	8	9	10	11
1. Intrinsic motivation	—
2. Perceived competence	.38**	—
3. Perceived difficulty	−.23**	−.63**	—
4. Valuing	.37**	.37**	−.15**	—
5. Devaluing	−.66**	−.38**	.29**	−.63**	—
6. Avoidance	−.65**	−.43**	.39**	−.37**	.69**	—
7. Dedication	.56**	.49**	−.32**	.49**	−.56**	−.66**	—
8. Reading importance support	.06	.15**	−.02	.39**	−.23**	−.05	.21**	—
9. Collaboration support	.21**	.29**	−.13**	.32**	−.25**	−.20**	.29**	.39**	—
10. Choice support	.12**	.19**	−.09*	.30**	−.21**	−.08*	.16**	.48**	.48**	—
11. Competence support	.07	.13**	−.03	.33**	−.21**	−.06	.19**	.49**	.43**	.47**	—
Mean	2.59	3.09	2.02	2.84	2.07	1.75	3.11	2.78	3.06	2.85	2.60
Standard deviation	.94	.50	.58	.64	.71	.64	.56	.63	.58	.60	.63
N	587	587	587	587	586	587	586	585	586	582	586

^*p < .05.

^**p < .01.

TABLE 5.

Correlations, Means (Ms), and Standard Deviations (SDs) of Motivations and Engagement Practices at Times 2 and 3

	1	2	3	4	5	6	7	8	9	10	11
1. Intrinsic motivation	—	.38**	−.20**	.51**	−.67**	−.68**	. 57**	.27**	.15**	.24**	.15**
2. Perceived competence	.33**	—	−.69**	.35**	−.29**	−.41**	.51**	.18**	.20**	.26**	.18**
3. Perceived difficulty	−.21**	−.72**	—	−.15**	.15**	.32**	−.30**	−.09*	−.20**	−.18**	−.12**
4. Valuing	.46**	.35**	−.14**	—	−.69**	−.49**	. 50**	.48**	.22**	.33**	. 32**
5. Devaluing	−.66**	−.33**	.21**	−.71**	—	.70**	−.55**	−.41**	−.27**	−.32**	−.28**
6. Avoidance	−.63**	−.34**	.30**	−.45**	.69**	—	−.71**	−.24**	−.28**	−.28**	−.14**
7. Dedication	.54**	.41**	−.23**	.51**	−.56**	−.70**	—	.27**	.28**	.32**	.21**
8. Reading importance support	.14**	.18**	−.06	.41**	−.30**	−.16**	.22**	—	.41**	.51**	.60**
9. Collaboration support	.13**	.16**	−.06	.18**	−.21**	−.23**	.27**	.39**	—	.60**	.39**
10. Choice support	.22**	.22**	−.09*	.28**	−.26**	−.25**	.27**	.52**	.63**	—	.46**
11. Competence support	.10*	.20**	−.08	.35**	−.25**	−.16**	.28**	.62**	.44**	.50**
M time 2	2.54	3.06	2.02	2.77	2.10	1.76	3.09	2.72	3.14	2.94	2.49
SD time 2	.93	.53	.58	.63	.72	.66	.58	.71	.66	.65	.68
N time 2	590	581	592	591	595	594	594	588	593	586	593
M time 3	2.50	3.06	2.01	2.73	2.17	1.83	3.06	2.64	3.28	3.07	2.47
SD time 3	.94	.56	.60	.69	.74	.69	.61	.76	.58	.64	.73
N time 3	587	588	591	588	591	591	591	592	592	591	593

Note. The data for time 2 are shown in the lower left part of the chart and for time 3 in the upper right, split by — in the null cells.

^*p < .05.

^**p < .01.

Motivations correlated with the perceptions of instructional practices in consistent patterns in all three time periods. The motivations of intrinsic motivation, perceived competence, and value all correlated significantly and positively with the instructional practices of support for importance, collaboration support, choice, and competence support. Simultaneously, the motivations of perceived difficulty and devalue correlated negatively with all four of the engagement practices, as expected, although two were not statistically significant. At time 3 (see Table 5, upper right), avoidance correlated negatively with importance at −.24 (p < .01), collaboration support at −.28 (p < .01), choice at −.28 (p < .01), and competence support at −.14 (p < 0.01), whereas dedication, the affirming form of engagement, correlated positively with each engagement practice, including importance at .27 (p < .01), collaboration support at .28 (p < .01), choice at .32 (p < .01), and competence support at .21 (p < .01).

At time 2, shown in the lower left of Table 5, 25 of 28 correlations followed the pattern of time 3, although three correlations were not significant. At time 1, shown in Table 4, prior to any intervention, 23 of 28 correlations between motivational and engagement practices showed the identical pattern of time 3. Thus, across three time periods, the correlations of motivational and engagement practices were consistently significant in expected directions, displaying a slight increase from time 1 to time 3.

The second hypothesis was that increases in students’ motivations and engagement would be more highly associated with the motivational-engagement supports in CORI than in TI. In the main analysis to test this hypothesis, we formed a composite of motivational-engagement supports by summing the scores of instructional supports for importance, autonomy, collaboration, and competence, which were correlated at .39–.62 at both time 2 and time 3, as shown in Table 5. For the dependent variables of motivation and engagement, we formed a composite of intrinsic motivation, self-efficacy, value, and dedication, which were correlated at .33–.57, as shown in Table 5, to represent a general academic motivational construct. Note in this composite that we did not include the undermining motivations of perceived difficulty, devaluing, or the negative version of engagement consisting of avoidance. Our rationale for this composite was that motivational-engagement supports in CORI were not designed to influence the undermining variables but were targeted to increase levels of affirming variables. However, for supplementary descriptive purposes, we investigated the effects of the set of motivational-engagement practices on both affirming and undermining motivations and engagements separately in the following analysis. We expected that the motivational-engagement supports would influence students’ motivations and engagement more strongly in CORI than in TI.

The statistical tests consisted of conducting multiple regressions with the motivation composite at a later time as the dependent variable (e.g., time 2). Independent variables consisted of gender, ethnicity, the motivation composite at an earlier time (e.g., time 1) as the autoregressor, and the set of instructional motivational-engagement supports from the appropriate interval. We performed this analysis separately for CORI and TI conditions. We hypothesized that the motivational-engagement supports in CORI would show a higher β weight in predicting motivation and engagement than in TI. We compared CORI and TI twice, using assessment from time 1 to time 2 and from time 2 to time 3. The results are shown in Table 6.

TABLE 6.

Associations of Instructional Practice Sets With Composite Motivation Scores

	β	Standard error β	β	R2 change
From time 1 to time 2
Concept-Oriented Reading Instruction	.263	.048	.25	.055***
Traditional instruction	.154	.035	.16	.023***
From time 2 to time 3
Concept-Oriented Reading Instruction	.262	.048	.25	.058***
Traditional instruction	.161	.042	.16	.021***

^***p < .001.

In Table 6, it is evident that from time 1 to time 2, the β for the effect of CORI motivational practices was .25, explaining 5.5% of the motivation-engagement variance, which was significant at p < .001. From time 1 to time 2, TI had a β of .16, explaining 2.3% of the motivation-engagement variance, which was significant at p < .001. The difference of the two βs was statistically significant, with z = 1.85 (p < .03, one-tailed; p < .06, two-tailed). As the hypothesis was directional, we accepted the probability of the one-tailed test. From time 2 to time 3, the β for the effect of CORI motivational and engagement practices was .25, explaining 5.8% of the variance, which was significant at p < .001. In this time frame, TI had a β of .16, explaining 2.1% of the variance, which was also significant at p < .001. For this second period, the difference of the two βs was statistically significant, with z = 1.60 (p < .05, one-tailed; p < .11, two-tailed). Again, we accepted the one-tailed test, given the directional hypothesis. Note that the probability of the two results both favoring CORI is the product of the two individual probabilities, which were p < .0015 for the one-tailed test, and p < .0066 for the two-tailed test.

The results confirm the expectation that motivational and engagement support practices within CORI were stronger than those practices inside TI for increasing motivation and engagement. The effect sizes are represented by the β weights, indicating that CORI showed increases of about .25–.26 standard deviations and that TI showed increases of about .15–.16 standard deviations. This is remarkably similar to the effect size of .26 showing the CORI advantage over TI for informational text reading comprehension. We interpret this to be a statistically significant, moderate effect on motivational-engagement practices on the motivation composite because of the relatively brief intervention, general motivation measures, and realistic classroom context of implementation.

As a supplementary analysis, we performed multiple regression analyses separately for CORI and TI at both treatment times that examined the associations of individual motivational and engagement practices with the individual motivations. For each analysis, the dependent variable was a motivation at a later time (e.g., intrinsic motivation at time 2), and the autoregressor was the same motivation at an earlier time (e.g., intrinsic motivation at time 1). Additional independent variables included gender and ethnicity. The final set of independent variables included the four individual motivational and engagement practices, consisting of supports for importance, collaboration, choice, and competence. Table 7 shows the results and lists all dependent and independent variables.

TABLE 7.

Associations of Individual Motivational-Engagement Support Practices With Individual Motivation and Engagement Constructs at Two Timepoints for Concept-Oriented Reading Instruction (CORI) and Traditional Instruction (TI) Separately

	Intrinsic motivation	Perceived competence	Valuing	Dedication	Perceived difficulty	Devaluing	Avoidance
From time 1 to time 2
CORI
All practices ΔR2	.02*	.04**	.04***	.03**	.01	.02	.02**
Reading importance support	.10	.02	.20**	.05	−.05	−.16***	−.09
Collaboration support	−.09	.02	−.04	.08	−.02	.01	−.08
Choice	.07	.14**	−.01	.02	.04	.01	−.06
Competence support	−.01	.10	−.08	.08	−.07	.02	.02
TI
All practices ΔR2	.01	.03**	.05***	.02**	.01	.01	.02**
Reading importance support	−.08	−.02	.11	−.08	.02	−.03	.02
Collaboration support	−.04	.11	−.05	.02	.01	.01	−.02
Choice	.13*	.06	.15**	.02	−.11	−.05	−.11**
Competence support	.04	.06	.06	.15**	.00	.05	.06
From time 2 to time 3
CORI
All practices ΔR2	.02***	.03***	.07***	.04***	.03***	.05***	.02***
Reading importance support	.11*	.09	.27***	.08	−.02	−.12*	−.11
Collaboration support	.01	.01	.08	.15**	−.10	−.11	−.15**
Choice	.05	.16**	−.04	.07	−.06	.02	.00
Competence support	−.06	−.11	−.07	−.11	−.02	.02	.10
TI
All practices ΔR 2	.01*	.01	.11***	.03*	.01	.07***	.03**
Reading importance support	.12*	.01	.26***	.06	.00	−.14***	−.05
Collaboration support	.02	.05	−.05	−.03	−.13	.00	−.03
Choice	.02	.05	.17**	.13*	.02	−.09	−.13**
Competence support	−.04	−.03	.02	.02	.00	−.08	.02

Note. Values in this table are βs from final models with demographic and autoregressor controls.

^*p < .05.

^**p < .01.

^***p < .001.

From time 1 to time 2, the set of practices in CORI was associated with significant changes on five motivation-engagement variables at p < .05, whereas the set of practices in TI was associated with significant changes on four motivation-engagement variables at p < .05. From time 2 to time 3, the set of practices in CORI was associated with significant changes on seven motivation-engagement variables at p < .001, whereas the set of practices in TI was associated with significant changes on two motivation-engagement variables at p < .001. The prevailing outcome is that the cluster of all motivational-engagement practices increased growth for the majority of motivation-engagement constructs at both times and for both instructional conditions. Importantly, however, significant effects occurred for more outcome variables in CORI than in TI. This supplementary analysis is consistent with the results of the main analysis shown in Table 6. In addition, across both CORI and TI, individual motivational-engagement supports of emphasizing importance and providing choice were significantly associated with motivation changes more frequently than were other practices.

Discussion

We observed that participation in CORI, which combined a composite of motivational-engagement supports with strategy instruction for informational texts in history, increased comprehension of informational text for middle school students more strongly than TI did. Because the study used the switching replications design, all students were compared with themselves, and individual differences were controlled experimentally (Shadish et al., 2002). This design permits the relatively strong inference that the experimental composite of CORI contributed to students’ increases in informational text comprehension. The finding is not attributable to fluency because it was controlled in the analyses, and fluency as an outcome variable showed inconclusive results.

According to Cohen (1977), effect sizes below .20 are small, those between .21 and .50 are moderate, and those higher than .50 are large. A meta-analysis of CORI studies (Guthrie et al., 2007) showed that effect sizes of CORI for informational text comprehension averaged .73. The meta-analytical studies were longer than the current study, as they lasted from six weeks to one year, and they involved younger students, ranging from grade 3 to grade 5. In this light, the low-moderate effect size of .26 found in this study seems realistic for a one-month implementation with middle school students assessed on a complex, higher order reading comprehension test.

In identifying the sources of the effects of CORI in the present study, it is important to consider the similarities and differences of CORI and TI, as shown in Appendix F. Similarities included class size, class time for reading/language arts, availability of texts and media, writing activities, and general reading comprehension goals. Students and teachers were identical in CORI and TI due to the within-subjects experimental design.

Differences between CORI and TI included the subject matter, instructional texts, strategy instruction, and motivational-engagement support. Each of these differences likely contributed to higher student performance during CORI; we believe, however, that the enhanced motivational-engagement support was the decisive factor. When students experienced relevance, personal meaning, competence in handling complex text, and shared interpersonal relationships, they were energized to process the structures and connections in informational texts relatively deeply. Students persevered in unraveling complex text, organized it into their knowledge structures, and constructed a viewpoint for their culminating projects. Lacking these supports, TI students were more likely to read more literally and remember texts in more fragmented forms.

In CORI, direct instruction of cognitive strategies for reading informational texts was inextricably integrated with motivational-engagement support. Although it might be suggested that strategy instruction primarily accounts for the results of the study, prior investigations (Guthrie et al., 2004) showed that strategy instruction in isolation did not increase informational text comprehension more than TI did, and strategy instruction had less influence on comprehension than CORI did, making this explanation improbable. Due to prior evidence that CORI in the form of a science–literacy integration increased informational text comprehension and standardized reading comprehension (Guthrie et al., 2004), it is unlikely that history uniquely links to CORI to increase comprehension. Also contributing to the CORI benefit was the experience of reading the informational texts themselves, as CORI students read more texts of this type than TI students did.

In sum, we propose that instructional differences, consisting of motivational-engagement support, history texts, and strategy instruction, all more present in CORI than in TI, were accountable for the observed increases in history text comprehension for these middle school students.

It is possible to claim that content of the informational text comprehension measure in this study was more aligned to CORI than to TI and, consequently, that the CORI advantage is unsurprising or even trivial. However, the purpose of this study was to investigate effects of CORI on informational text comprehension in the domain of history, and thus an informational text comprehension measure in history was necessary for this purpose. Specific topics of the assessment were not present in either the CORI or the TI units. CORI focused on the causes and outcomes of the Civil War, whereas the assessment passages concerned such topics as the Revolutionary Army, Quaker beliefs, battleships, military prisons, and black soldiers (see Appendix D). The probability is low that knowledge generated in CORI is responsible for the higher test performance. However, it is known that comprehension processes in history, such as perceiving text structures and contextualization, are general to a domain such as history (for a review, see Poitras & Lajoie, 2013). We expect that during CORI, students learned these processes and transferred them to other reading topics in the informational text comprehension assessment.

The present data constrain us to the conclusion that CORI was higher than TI in history text comprehension. This finding is not inconsequential due to the importance of disciplinary literacy for school learning. We believe that the three parallel forms in the outcome measure required sufficiently broad, deep, challenging, and generalizable text comprehension to make the finding worthy of note to the field. In sum, we suggest that the CORI constituents of motivational-engagement support, history texts, and strategy instruction were accountable for increasing history text comprehension in middle school students. This conclusion is consistent with findings from a meta-analysis of 11 studies involving 75 comparisons. These comparisons demonstrated that CORI was significantly higher than TI in performance on informational text comprehension measures in science, as well as standardized reading comprehension tests and literary comprehension measures (Guthrie et al., 2007).

These results extend the findings from previous, more task-specific experimental studies of motivation and engagement. A valuable, although relatively small, set of studies used experimental designs to examine instructional effects on motivation or engagement in short-term conditions (Jang, 2008; Vansteenkiste, Simons, Lens, Soenens, & Matos, 2005). Under these settings, instructions for performance in a specific task situation may evoke an existing motivation that is limited to time and task. Consequently, it is unknown whether the instruction in these studies increased motivation in the long term for a range of tasks in the domain. Our findings suggest that multiple motivational-engagement supports combined with strategy instruction for informational text increased achievement in an educationally significant classroom unit for middle school students.

We found that the set of four motivational-engagement practices was associated with a relatively high increase in a composite of motivation and engagement constructs. Specifically, students’ perceptions of motivational-engagement supports in CORI explained substantially more variance in the motivation-engagement outcomes than the same perceived supports in TI. Drawing on experimenter-provided professional development, CORI teachers explicitly provided tasks, activities, and class discussions related to students’ choices, collaboration, importance, and competence during reading instruction. Although teachers were free to employ these practices during TI based on their judgment and preferences, they were relatively less likely to influence change in motivation or engagement when employing them. It should be recognized that the motivational-engagement supports were accompanied in CORI by strategy instruction for comprehending informational texts, and historical materials related to the Civil War, which may have contributed to students’ motivation changes in concert with the motivational-engagement practices; these accompaniments, however, were unmeasured, and their effects are unknown.

The first motivational-engagement support was affording students’ autonomy or choice and input into instruction as articulated in self-determination theory (Assor, Kaplan, Kanat-Maymon, & Roth, 2005). Choice has rarely been studied as an experimental variable in classroom motivation studies, although Patall, Cooper, and Wynn (2010) showed positive effects of homework choice on intrinsic motivation and engagement in the form of completing assignments. The current investigation suggests the benefits of choice combined with other motivational-engagement supports for increasing a composite of motivations. Second, the classroom activities that are intended to support students’ values for reading by emphasizing the importance, benefit, and usefulness of reading have not frequently been investigated in experiments with K–12 students. The current findings affirm the potential of emphasizing importance as a useful contributor to motivational-engagement supports in the classroom. Third, this study confirmed the potential effects of competence support (Schunk & Mullen, 2012) on perceived competence and informational text comprehension for middle school students. Fourth, the motivational-engagement support of collaboration in literacy activities appears to be associated with advances in reading engagement, as expected from activity theory (Leontiev, 1981).

We observed that two forms of engagement, dedication and avoidance (Skinner et al., 2009), were not simply mirror opposites, because they correlated moderately and formed two factors in a factor analysis. The occurrence of these twin engagement forms is due to a sizable proportion of students reporting that they are both dedicated to reading and avoidant of reading. Students may be dedicated (committing time and effort) for some informational books in some situations and be avoidant (evasive of reading) for other informational books in other situations. For instance in a Civil War unit, a student could spend time reading a biography of Harriet Tubman enthusiastically while actively minimizing her interaction with texts on the battles of Fort Sumter and Gettysburg.

As shown in the correlations for all time periods, dedication, the affirming form of engagement, was positively predicted by three affirming motivations (intrinsic motivation, value, and perceived competence) and negatively predicted by one undermining motivation (devalue), which extends previous studies of the relations of motivations to engagement (Reeve, 2012). At the same time, avoidance, the undermining form of engagement, was simultaneously positively predicted by two undermining motivations (perceived difficulty and devalue) and negatively by one affirming motivation (intrinsic motivation), which extends work on undermining motivation constructs (Legault et al., 2006). This suggests that there may be psychological differences between dedication and avoidance, which merit further research.

Previous studies of CORI used an equivalent-groups quasi-experimental design to compare CORI and control groups (Guthrie et al., 2007), showing substantial effect sizes for reading comprehension. Although improbable, this design may allow unknown and unmeasured variables to be confounded with the instructional treatment. In the present study, the switching replications experimental design, in which each person’s learning is compared under both CORI and control conditions, adds certainty to the previous findings. Additionally, prior CORI studies did not attempt to determine the association of any single instructional practice, such as providing choices, on outcome variables of motivation or achievement. In the present study, each of four instructional practices showed a theoretically expected association with motivation change. A practical implication is that combined with more reading of complex text and strategy instruction for that reading, increasing the multiple engagement supports for academic literacy may be beneficial for informational text comprehension.

There were several limitations to this study, including the age group of middle school students and the length of the instructional intervention, which was one month. Although the effects may vary if any of these study characteristics were altered, it is reasonable to expect that the findings may generalize to teaching that employs informational texts, age groups from 8 to 15 years old, and instructional units of four to eight weeks. The sample was not representative of the state or district, but there was no selection bias into treatments due to the switching replications design.

Another limitation was the use of a constrained set of motivation constructs, consisting of intrinsic motivation, perceived competence, value, perceived difficulty, and devalue. Other motivations, such as interest, mastery goals (Pintrich, 2000), or prosocial goals, could also be investigated. Likewise, a limitation was the set of motivational-engagement supports that focused on support for intrinsic motivation, perceived competence, value, and behavioral engagement. It is possible to investigate the effects of support for emotional relationships through teacher–student interactions (Wentzel, 2010) or to support students’ mastery goals through classroom goal structures (Andermann, Andrezejewski, & Allen, 2011). It would be valuable to investigate whether these social and goal-oriented constructs contribute to academic engagement beyond the existing set of motivational-engagement supports examined in this study.

We did not attempt a mediation analysis that would permit us to determine whether CORI ’s effects on informational text comprehension were attributable to the effects on motivation or engagement as shown by Wigfield et al. (2008). Another limitation is that the motivational-engagement supports in this intervention were placed in the subject matter of the Civil War. It is doubtful that this combination was unique because these identical motivational-engagement supports showed similar effects for middle school students in biodiversity (Guthrie et al., 2013). Finally, sheer reading volume was possibly higher in CORI than in TI, which may have increased informational text comprehension performance. However, this is doubtful because previous studies showed that when motivation and engagement were controlled as they were in this study, amount of reading did not increase reading achievement (Guthrie et al., 2004).

APPENDIX A. Items on Motivation and Engagement Scales

The directions as follows were read aloud to students.

This questionnaire asks about your reading of information books for school. Information books are any books that tell you real facts and knowledge. School reading is any reading that will help you in school. It does not have to take place in your school building. School reading can be homework reading or studying, too. You will read each of the sentences and then circle your answer choice. The answer choices are very true of me, somewhat true of me, not very true of me, or not at all true of me. There are no right or wrong answers. It is very important to be honest in your answers. This information will not be given to your parents or teachers. It will be used only by the university research team to help them understand more about students’ reading. You will have enough time to think about each of the items.

Then, a sample question was given.

	Excluded?	Factor loading
Intrinsic motivation (Cronbach’s α = .72, N = 1,053)a
Definition: Enjoyment of reading and having a desire to read often.
The reading materials were interesting.	X	.13 on F1; .90 on F2
It was fun to do the readings.	X	.29 on F1; .86 on F2
During my free time, I read often.		.88 on F1
I read as much as I could.		.82 on F1
I read for long periods of time.		.86 on F1
I enjoyed reading.		.66 on F1
Perceived competence (Cronbach’s α = .78, N = 1,049)
Definition: The individual’s belief about his or her capacity to successfully complete reading tasks.
I could understand all the readings.		.63
I could correctly answer questions about the readings.		.69
The key points in the text were clear to me.		.79
The main ideas of the readings were easy to find.		.74
I could figure out what unfamiliar words meant.		.63
I figured out how different chapters fit together in the readings.		.66
Perceived difficulty (Cronbach’s α = .81, N = 1,050)
Definition: The individual’s perception that reading is hard.
The reading materials were way too hard for me.		.65
The readings were really confusing to me.		.76
It was hard for me to discuss the reading materials.		.76
I had a hard time explaining to classmates what the texts meant.		.69
The teacher’s questions about the readings were hard for me to answer.		.77
I needed help understanding some of the main ideas.		.65
Value (Cronbach’s α = .79, N = 1,059)
Definition: Belief in the importance and usefulness of reading.
The readings gave me useful knowledge.		.75
I could relate the readings to my life.		.57
It was very important to me to do my reading.b	X	.70
Studying the materials was beneficial to me.		.78
Understanding the reading materials will help me next year.		.73
I learned something valuable from the reading assignments.		.80
Devalue (Cronbach’s α = .81, N = 1,055)
Definition: Belief that reading is not important or useful for one’s success or future.
I could not relate to the readings.b	X	.49
Reading the materials was not useful for me.		.75
Reading was a waste of time.		.83
Reading was not important to me.		.80
I had more important things to do than read.		.72
What I read will not help me in the future.		.60
Dedication (Cronbach’s α = .81, N = 1,064)
Definition: Time, effort, and persistence put into reading.
Even if the reading assignments were difficult, I completed them.		.62
I went above and beyond what was expected of me in reading.		.69
I spent as much time as needed to complete my reading homework.		.72
For every reading assignment, I worked hard.		.79
I made sure I had enough time to complete my reading assignments.		.73
I put a lot of effort into reading.		.74
Avoidance (Cronbach’s α = .85, N = 1,061)
Definition: Having an aversion toward reading and therefore minimizing time and effort spent on reading.
I did not try to complete most of the reading assignments.		.62
I read as little as possible.		.74
Every day, I tried to get out of reading.		.80
I put as little effort as possible into my reading.		.75
I avoided spending time on the readings.		.81

^aOnly construct comprised of two factors when each of seven constructs individually factor analyzed; two items on second factor dropped.

^bItem dropped because it loaded more strongly with “opposite” construct when value/devalue items factor analyzed together.

APPENDIX B. Informational Text Comprehension Assessment

Topics From Three Parallel Forms

Form A

• Ironclad Battleships

• The Quakers

• Black Volunteers [linking passage]

• Letter From General Hood

• Society of Cincinnati in 1783

Form B

• Harpers Ferry Raid

• Military Prison

• Black Volunteers

• Vigilance Committees in Tennessee

• Continental Army in Valley Forge

Form C

• Merrimack Description

• Thomas Elwood Rose and the Pennsylvania Flag

• Black Volunteers

• Evacuation of Atlanta

• Continental Army Drill

Example Passages and Questions

The question type appears in brackets following each question stem.

Lower Difficulty

The Quakers

The Quakers are a Christian religious group that was started by George Fox, an Englishman, in 1648. Equality is a very important part of the Quaker way of thinking. Because Quakers were such strong believers in equal rights for all people, they were strongly opposed to slavery. In the 1800s, many Quakers were also abolitionists. They helped slaves escape and offered their homes, stores, stables, and barns as “stations” on the Underground Railroad. They risked their own safety and freedom to help other people who were not free. There are still Quakers today. They worship in Quaker meeting houses without ministers or priests. Instead, each member of the group is allowed to speak at the meeting.

• 6Quakers risked their own safety and freedom because they [phrase understanding]

  1. wanted to worship together freely.

  2. thought others should be free as well.

  3. were founded by a Christian in England.

  4. were opposed to the abolition of slavery.

• 7Based on the passage, many Quakers were probably [literal understanding]

  1. politicians.

  2. slaves.

  3. farmers.

  4. ministers.

• 8In this passage, stations refers to places where [word in context]

  1. trains deposit and collect passengers.

  2. slaves could find jobs.

  3. escaped slaves could hide.

  4. Quakers were allowed to worship.

• 9All Quaker members are allowed to speak at meetings, which is an example of the Quakers’ belief in [basic conceptual understanding]

  1. equality.

  2. slavery.

  3. abolitionism.

  4. helping people.

• 10According to the passage above, the responsibilities of an abolitionist included [literal understanding]

  1. helping escaped slaves find places to stay.

  2. worshiping in Quaker meeting houses.

  3. preparing meals for escaped slaves.

  4. finding escaped slaves in the North.

Harder Passage

Baron de Steuben and the Society of Cincinnati

Many in Congress viewed the Society of Cincinnati as an insidious cabal “formed in Europe to overturn our happy institutions.” One of them was Aedanus Burke of South Carolina. Under the pseudonym “Cassius,” Burke wrote a pamphlet attacking the Society as a “race of hereditary patricians or nobility,” singling out Baron de Steuben as the “creator” and “Grand Master” of the order. “I have the honor to inform Baron Steuben” he wrote, “that an order of peerage may do well under the petty princes of Germany, yet, in America, it is incompatible with our freedom.”

Steuben, who joined the New York chapter of the Cincinnati in 1786 and served for several years as its president, shrugged off the ridiculous accusations. “A ca, Monsieur le Cincinnatus,” he wrote in jest to Henry Knox, the real guiding hand behind the Society, in November 1783, “Your pernicious designs are thus revealed. You wish to introduce dukes and peers into our Republic. No, my Lord, no, my Grace, that will not do….Blow Ye the Trumpet in Zion!”

Laugh as he might at the carpings of the “Bostonians and gentlemen of the Holy Land” and their “modest and Presbyterian airs,” the truth was that the Baron’s position damaged his standing with Congress. Massachusetts delegate Rufus King jabbed at him:

I know that he was a Soldier of Fortune and a mercenary in Europe; and notwithstanding his affected philanthropy and artificial gentleness, I hold his character the same in America; the only difference is this: in Europe he received little money and less flattery….He has from this circumstance of preference and from the adulation of the sycophants, been buoyed up to the preposterous belief that his military talents are superior to those of any soldier in America.

• 21Which of the following statements about Baron de Steuben is true based on the passage above? [subconcept]

  1. Baron de Steuben became the first elected president of the Society of Cincinnati.

  2. Baron de Steuben was fervently opposed to American religious institutions.

  3. Baron de Steuben lost political prestige despite maintaining social influence.

  4. Baron de Steuben disagreed with Henry Knox’s plans for the Society of Cincinnati.

• 22Which of the following statements best summarizes the second paragraph of the above passage? [partial passage synopsis]

  1. Baron de Steuben was closely involved in the Society of Cincinnati and did not place much importance on the opinion of Congress.

  2. Henry Knox was one of the leading supporters of the Society of Cincinnati, often having to defend the Society’s position before Congress.

  3. Members of Congress wrote to Baron de Steuben, forbidding him from admitting European nobility into the Society of Cincinnati.

  4. Baron de Steuben was president of the Society of Cincinnati and relied heavily on the advice of Henry Knox, the previous president.

• 23What is the main idea of this passage? [main concept]

  1. Baron de Steuben and Henry Knox were among the Europeans who came to America and founded the Society of Cincinnati in 1786.

  2. Congress wanted to eliminate the Society’s New York chapter because they did not approve of the society’s methods.

  3. The Society of Cincinnati provided Baron de Steuben the prestige, wealth, and power he had always strived to attain.

  4. Baron de Steuben’s political influence was weakened by his connection to the Society of Cincinnati.

• 24Which of the following statements best summarizes the entire passage? [full passage synopsis]

  1. The Society of Cincinnati was repeatedly discredited by several members of Congress as being exclusionary and elitist, even though the society claimed that it was open to all who wished to apply, regardless of status.

  2. Baron de Steuben was the target of many members of Congress due to his vehement belief in the intellectual superiority of nobility and his stance against the religious practices employed by those members of Congress.

  3. Although Congress disagreed with the patriarchal nature of the Society of Cincinnati and questioned the motives of Baron de Steuben, he mocked their position and continued to lead the New York Chapter.

  4. Congress feared that the Society of Cincinnati would bring the patriarchal views of Europe to America and began a campaign that would ultimately tarnish the reputation of Baron de Steuben within both Congress and the military.

• 25One delegate theorized that one reason for Baron de Steuben’s arrogance was the baron’s [relational understanding]

  1. leadership role in the Society of Cincinnati.

  2. lineage of German nobility.

  3. high salary in the U.S. military.

  4. position of power within Congress.

APPENDIX C. Unit and Weekly Frameworks for CORI

Unit Framework

	Week 1	Week 2	Week 3	Week 4
Content	Causes of the Civil War: A Nation Divided	Causes of the Civil War: Unity vs. States’ Rights	Military Conduct and Events of War	Impact of War
Concepts	Economics, culture, slavery, politics	Leadership, beliefs	Military	All
Comprehension instruction	Inferencing	Summarizing	Concept mapping	Multiple strategies
Motivation	Success	Choice	Reading importance	Collaboration
Whole-class text	The Causes of the Civil War	The Causes of the Civil War	Key Battles of the Civil War	Reconstruction and the Aftermath of the Civil War
Guided reading – OGL	A Nation Divided: Causes of the Civil War	The Home Front in the North and The Home Front in the South	The Civil War in the East and Key Battles of the Civil War	Reconstruction and the Aftermath of the Civil War
Struggling readers	Secession	Key People of the Civil War	Battles of the Civil War	Reconstruction and the Aftermath of the Civil War and The Carpetbaggers
Advanced readers	CORI informational text enrichment	CORI informational text enrichment	CORI informational text enrichment	CORI informational text enrichment

Weekly Topic — Causes of the Civil War: A Nation Divided

	Lesson 1	Lesson 2	Lesson 3	Lesson 4	Lesson 5
Social studies content/concepts	A Nation Divided: Economics: What were the similarities and differences in the economies of the North and the South?	A Nation Divided: Culture: What were the key aspects of Northern and Southern cultures?	A Nation Divided: Slavery: How do you explain the different views about slavery in the North and the South?	A Nation Divided: Politics: How did political policy impact the country before the Civil War?	A Nation Divided: Politics (Secession): Why did the Southern states secede from the Union?
Knowledge goals(via video)	Northern Cities, Southern Plantations	Life in the Northern and Southern States	Life on Southern Plantations	Political Power Shifts	The Confederate States of America
Comprehension instruction	Inferencing: Guide questions 1 & 2	Inferencing: Guide questions 1 & 3	Inferencing: All guide questions	Inferencing: All guide questions	Inferencing: All guide questions
Motivation	Success: How did the video help you feel confident in your reading?	Success: What are some characteristics of the text you read today that helped you succeed as a reader?	Success (exit slip): How did your success with inferencing motivate you to read today?	Success: How did improving your inferences help you understand the text better?	Success: How did improving your inferencing make you a more confident reader?
Whole-class text	The Causes of the Civil War	The Causes of the Civil War	The Causes of the Civil War	No whole-class instruction	No whole-class instruction
Guided reading – OGL	The Causes of the Civil War	A Nation Divided: Causes of the Civil War (H) and The Causes of the Civil War (L)	A Nation Divided: Causes of the Civil War (H) and The Causes of the Civil War (L)	A Nation Divided: Causes of the Civil War (H) and The Causes of the Civil War (L)	A Nation Divided: Causes of the Civil War(H) and The Causes of the Civil War (L)
Struggling readers	Secession	Secession	Secession	Secession	Secession
Advanced readers	A Nation Divided: Causes of the Civil War	CORI informational text enrichment	CORI informational text enrichment	CORI informational text enrichment	CORI informational text enrichment
Writing	Writing facts and concepts, inferences	Writing facts and concepts, inferences	Writing facts and concepts, inferences	Writing facts and concepts, inferences	Writing facts and concepts, inferences
Independent reading	Informational text reading: Influential people and special groups of the Civil War

APPENDIX D. Reading and Writing Objectives in the Grade 7 Voluntary State Curriculum for the Period of the CORI Intervention

Week 1 Reading: General

• Acquire new vocabulary through listening to, independently reading, and discussing a variety of literary and informational text.

• Classify and categorize increasingly complex words into sets and groups.

• Explain relationships among words.

• Use vocabulary in speaking and writing to gain and extend content knowledge and to clarify expressions.

Week 2: Reading

• Show attendance to sentence patterns and structures that signal meaning in text, using punctuation cues to guide meaning and expression.

• Use vocabulary in speaking and writing to gain and extend content knowledge and to clarify expression.

• State and support main ideas and messages.

• Identify and explain the main idea.

• Identify and explain information directly stated in the text.

• Summarize or paraphrase.

• Connect the text to prior knowledge or experience.

• Explain relationships between and among ideas.

• Synthesize ideas from text.

Writing

• Select, organize, and develop ideas appropriate to topic, audience, and purpose.

• Use writing to learn strategies such as reflective and metacognitive writing to set goals, make discoveries, and make connections among learned ideas.

Week 3: Reading (see Appendix E)

• Discuss reactions to and ideas gained from reading with adults and peers in formal and information situations.

• Select and apply appropriate strategies to prepare for reading text.

• Select and apply appropriate strategies to make meaning from text during reading.

Writing

• Locate, retrieve, and use text from a variety of sources to accomplish a purpose.

• Use writing to learn strategies such as reflective and metacognitive writing to set goals, make discoveries, and make connections among learned ideas.

Week 4: Comprehension of Informational Text

• Apply comprehension skills by selecting, reading, and interpreting a variety of print and nonprint informational texts, including electronic media.

• Analyze important ideas and messages in informational text.

• Read critically to evaluate informational text.

• Read orally at an appropriate rate.

• Read grade-level text with both high accuracy and appropriate pacing, intonation, and expression.

• Develop and apply vocabulary through exposure to a variety of texts.

• Apply a conceptual understanding to new words.

• Understand, acquire, and use new vocabulary.

• Apply comprehension skills through exposure to a variety of print and nonprint texts, including traditional print and electronic texts.

• Use strategies to prepare for reading.

• Use strategies to make meaning from text.

• Use strategies to demonstrate understanding of text.

Writing

• Compose texts using the prewriting and drafting strategies of effective writers and speakers.

• Compose oral, written, and visual presentations that express personal ideas, inform, and persuade.

• Locate, retrieve, and use information from various sources to accomplish a purpose.

APPENDIX E. Booklist for CORI Middle School Civil War History Week 3

• Smolinski, D. (2001). Key battles of the Civil War. Chicago, IL: Heinemann. (32 pages; class set)

• DeAngelis, G. (2003). The battle of Gettysburg: Turning point of the Civil War. Mankato, MN: Bridgestone. (48 pages; on grade)

• Fay, G. (2011). Battles of the Civil War. Chicago, IL: Heinemann. (48 pages; struggling reader)

• Excerpts of articles from history magazines (19 pages total; large format, fine print, 8.5″ × 11″; advanced readers; articles adapted for research purposes only):

  • “John Brown’s Raid on Harpers Ferry” (3 pages)

  • “The People at the Polls” (6 pages)

  • “Jefferson Davis” (3 pages)

  • “Voices of Secession: Edmund Ruffin” (2 pages)

  • “State’s Rights” (2 pages)

  • “The Declaration of Causes of Seceding States: Georgia” (3 pages)

• Stanchak, J.E. (2000). Civil War. New York, NY: Dorling Kindersley. (64 pages; supplemental; mixed levels of difficulty)

• Gunderson, C.G. (2002). Jefferson Davis: Let freedom ring. Mankato, MN: Bridgestone. (48 pages; one example of 20 biographies available for self-selected, required reading)

APPENDIX F

SIMILARITIES and Distinctions Between CORI and TI

	CORI	TI
Similarities
Class size	22–28 students
Class time	90 minutes daily
Available texts and media	Identical books and computers
Students	Identical
Teachers	Identical
Writing	Daily note taking, persuasive
General goals	Text comprehension, critical analysis
Differences
Content	Civil War and the Reconstruction	Literature with background history
Instructional texts	Civil War trade books, biographies	Anthology, documents
Strategy instruction	Inferencing, summarizing, concept mapping	Inferencing, plot tracing, character analysis
Motivational-engagement support	CORI teacher’s guide	Teacher initiated