Effects of Preventative Tutoring on the Mathematical Problem Solving of Third-Grade Students With Math and Reading Difficulties

Abstract

This study assessed the effects of preventative tutoring on the math problem solving of third-grade students with math and reading difficulties. Students (n = 35) were assigned randomly to continue in their general education math program or to receive secondary preventative tutoring 3 times per week, 30 min per session, for 12 weeks. Schema-broadening tutoring taught students to (a) focus on the mathematical structure of 3 problem types; (b) recognize problems as belonging to those 3 problem-type schemas; (c) solve the 3 word-problem types; and (d) transfer solution methods to problems that include irrelevant information, 2-digit operands, missing information in the first or second positions in the algebraic equation, or relevant information in charts, graphs, and pictures. Also, students were taught to perform the calculation and algebraic skills foundational for problem solving. Analyses of variance revealed statistically significant effects on a wide range of word problems, with large effect sizes. Findings support the efficacy of the tutoring protocol for preventing word-problem deficits among third-grade students with math and reading deficits.

Solving word problems, which is a major component of the primary-grade mathematics curriculum, is a form of mathematical cognition that is distinct from calculation skill (e.g., L. S. Fuchs, Fuchs, Stuebing, Fletcher, Hamlett, & Lambert, 2006). Although word problems require correct calculation for solution, they differ from calculation problems because of the addition of linguistic information. Whereas a calculation problem is already set up for solution, a word problem requires students to construct a problem model by identifying what information is missing, determining the number sentence that incorporates the given and the missing information, and deriving the calculation problem for finding the missing information. As might be expected, therefore, word problems are a source of difficulty for many students, especially those with disabilities, because these students often experience the language deficits that specifically account for mathematical problem-solving difficulty (L. S. Fuchs et al., 2006). Consequently, a need exists to identify effective prevention strategies to ameliorate word-problem deficits early, in the primary grades.

In recent years, most discussions about early prevention of academic difficulties have focused on multitier prevention systems (e.g., Vaughn & Fuchs, 2003). In the most common multitier prevention system, general education constitutes primary prevention. Students who fail to respond to general education, or the “universal” core program, enter secondary prevention. In most research studies, this involves one or more rounds of tutoring. Students who respond poorly to secondary preventative tutoring then become candidates for tertiary intervention. Tertiary intervention is an even more intensive form of instruction that typically involves individualized programming developed inductively via ongoing progress monitoring.

Most research on multitier prevention systems has focused on reading. Vellutino et al. (1996), for example, provided 16 weeks of daily tutoring for kindergarten and first-grade children with early signs of difficulty. The researchers estimated each child’s reading growth and, on that basis, classified children by degree of responsiveness, showing that the preventative tutoring substantially benefited many students. This study, together with other work (e.g., Al Otaiba & Fuchs, 2006; D. Fuchs, Compton, Fuchs, Bryant, & Davis, in press; McMaster, Fuchs, Fuchs, & Compton, 2005), illustrates how secondary preventative tutoring in reading can enhance student outcomes and reduce the likelihood of reading disability.

Less work has been conducted in math. Some studies have focused exclusively on basic facts or simple computation, with drill and practice used for brief tutoring intervention. VanDerHeyden and Witt (2005), for example, screened first and second graders in one school using basic fact assessments. For the subset of children who scored poorly, the researchers conducted nine tutoring sessions that involved feedback and reinforcement to increase basic fact fluency. More recently, L. S. Fuchs, Compton, et al. (2005) conducted a large-scale randomized controlled study of the effects of longer-term and more comprehensive preventive tutoring that addressed basic facts, computation, concepts, and word problems. Participants were 564 first graders in 41 classrooms in 10 schools. Of these students, 127 were designated at risk for math disability and randomly assigned to secondary preventative tutoring or to remain in their first-grade classrooms without tutoring. Tutoring occurred three times weekly for 20 weeks; treatment fidelity was documented, and math performance was assessed before and after tutoring. Tutored students performed better than control students on computation, concepts, and word problems. Moreover, secondary preventative tutoring decreased the prevalence of math disability, with effects maintaining to the end of second grade. This work supports the efficacy of secondary preventative tutoring on a comprehensive mathematics curriculum at first grade.

Beyond the first grade, however, less is known about the efficacy of secondary preventative tutoring in math. This is unfortunate because the 2004 reauthorization of the Individuals with Disabilities Education Improvement Act (Public Law 108–446) relies on a multitier prevention system, which incorporates preventative tutoring at the secondary tier of that prevention system. Specifically, the reauthorization permits states to switch from a traditional intelligence-achievement discrepancy model for identifying learning disabilities to a model of identification known as responsiveness to intervention, or RTI. The basic premise of RTI is that when a child responds poorly to validated secondary preventative tutoring, this eliminates instructional quality as a viable explanation for poor academic growth and instead provides evidence of disability. Secondary preventative tutoring must represent a validated form of instruction, to which most students can be expected to respond positively.

As schools struggle to implement RTI within a multitier prevention system, a need exists for validated protocols for conducting secondary preventative tutoring. The purpose of the present study was to explore the efficacy of a secondary preventative tutoring protocol on mathematics word problems at third grade. The target population was students with risk for mathematics and reading disability, because compared to students who experience mathematics difficulties without concomitant reading problems, the population of students with math and reading difficulties has more severe math deficits and is more similar to students identified with disabilities in the schools. In the remainder of this introduction, we describe previous research aimed at ameliorating math problem-solving difficulties; then we explain the instructional approach we incorporated into our secondary preventative tutoring protocol and present the purposes of the present study.

Beyond the first grade, however, less is known about the efficacy of secondary preventative tutoring in math.

PRIOR WORK

Much of the work designed to help students with learning difficulties enhance their mathematical word problem-solving skill has focused on the use of metacognitive planning and organization strategies. Montague and Bos (1986), for example, assessed the effects of an eight-step metacognitive treatment with 6 adolescents with learning disabilities. Students were taught to read problems, paraphrase the problems aloud, graphically display known and unknown information, state the known and unknown information, hypothesize solution methods, estimate answers, calculate answers, and check answers. Using a single-subject design, the researchers showed that this metacognitive treatment promoted word-problem skill. This approach has been extended for younger children, but with less evidence of benefits. For example, Shiah, Mastropieri, Scruggs, and Fulk (1994–1995) randomly assigned 30 students with learning disabilities, 16 of whom were in the primary grades, to two computer-assisted conditions that employed a seven-step metacognitive strategy and to one computer-assisted condition without the strategy. Results revealed no significant differences among the three treatment conditions.

The major contrasting approach in the research literature for developing word-problem skill for students with learning difficulties relies on schema theory. Although methods based on schema theory inevitably involve metacognition, they differ from the metacognitive approach because they explicitly teach students to group problems into types with similar underlying mathematical structures and teach students problem-solution rules for each problem type. As a major contributor to this literature, Jitendra and her colleagues have invoked schema-based instruction to enhance word-problem skill among students with low performance. With 4 eighth-grade students, for example, all of whom had serious math deficits and 2 of whom also experienced reading difficulties, Jitendra, DiPipi, and Perron-Jones (2002) used a multi-probe-across-participants design to examine effects on word problems requiring multiplication and division. Schemas for two problem types were taught, first by helping students distinguish multiplication from division problems and then by teaching problem–solution rules. Favorable acquisition and maintenance effects were achieved as well as generalization to novel one-step and multistep problems. To extend this study, Xin, Jitendra, and Deatline-Buchman (2005) conducted a randomized field trial with sixth- through eighth-grade students. Students’ math skill averaged approximately a standard deviation below the mean; their average reading performance was at or above the 25th percentile. Addressing the same two problem types, Xin et al. again documented impressive acquisition and maintenance as well as transfer for problems with irrelevant information and multiple steps.

Jitendra and colleagues have also demonstrated the efficacy of schema-based strategy instruction at the elementary grades. Using a multi-probe-across-participants design with 3 students in third or fourth grade, Jitendra and Hoff (1996) used a similar instructional method to teach one-step change, compare, and group problems, which required single-digit adding or subtracting for solution. Acquisition and maintenance effects were revealed. Although transfer was not addressed, effects were similarly strong across the 3 students; the math deficits of one of these 3 students were severe (reading performance was not reported). Jitendra, Griffin, McGoey, Gardill, Bhat, and Riley (1998) then extended this line of work by conducting a randomized controlled study with 34 students whose computational skill was adequate but who had specific difficulties with word problems (no information on reading performance was included). One student was in second grade; 5 were in third grade; 21 were in fourth grade; and 7 students were in fifth grade. Effects were strong on acquisition maintenance problems, and transfer occurred to problems not used for instruction.

More recently, Jitendra, Griffin, Haria, Leh, Adams, and Kaduvetoor (2007) tested the efficacy of their schema-based strategy instruction specifically at third grade, targeting change, group, compare, and two-step problems. They matched students into pairs on incoming Stanford math problem-solving scores as well as learning disability status, English language learner status, and math Title I status. They randomly assigned the 88 students, within pairs, to schema-based strategy instruction or to a metacognitive planning and organization treatment. Approximately half the students received reduced-price or free lunch; 4 students were identified as having a learning disability (only 1 student had a mathematics disability); reading performance was not reported. Intervention occurred in groups of 15 to 16 students for 8 to 9 weeks, 5 days per week, for 25 min per session. The sequence in which problem types were taught varied across the conditions. Schema-based strategy instruction addressed change, then group, then compare, and then two-step problems; by contrast, the metacognitive planning and organization treatment addressed addition word problems, then subtraction word problems, and then two-step problems. At the end of treatment and 6 weeks later, significant effects, in the moderate range, favored the schema-based strategy condition on the authors’ word-problem posttest that mirrored the problem types taught for instruction. On a state-administered test of mathematics performance, which assessed a variety of math skills including but not limited to problem solving, moderate effects also favored the schema-based strategy condition. Analyses were also conducted separately on a subset of students: 4 students with learning disabilities, 6 English language learners, and 11 students who received math Title I services (one of whom was also an English language learner, for a total of 20 students). Posttest effects for this subset of students did not favor schema-based strategy instruction, although at maintenance, a large, significant effect was detected, and a marginally significant effect on the state test also emerged.

In our own work on math problem solving, we have also relied on schema theory. We incorporate instructional features designed to meet similar goals as Jitendra et al. (2007). That is, we teach students to understand the underlying mathematical structure of the problem type, to recognize the basic schema for the problem type, and to solve the problem type. In contrast to Jitendra et al., however, we incorporate an additional instructional component by explicitly teaching students to transfer their problem-solving skills. We incorporate this additional, explicit transfer component given the hypothesis that transfer instruction, designed to broaden schemas, will help students recognize connections between problems like those worked during instruction and problems with unexpected features, such as problems that include irrelevant information, or that present a novel question requiring an extra step, or that include relevant information presented in charts or graphs, or that combine problem types, and so on. The hope is that the addition of explicit transfer instruction will lead to more flexible and successful problem-solving performance. We refer to the combination of all four instructional components (teaching students to understand the underlying mathematical structure of the problem, recognize the basic schema for a problem type, solve that problem type, and transfer their schemas to novel problems) as schema-broadening instruction (see Cooper & Sweller, 1987, for the theoretical basis for schema-related instruction that includes these four components).

In our first randomized controlled study, L. S. Fuchs, Fuchs, et al. (2003) separated the effects of the first three instructional components—teaching students to (a) understand the underlying mathematical structure of the problem type, (b) recognize the basic schema for a problem type, and (c) solve the problem type—from the last instructional phase (explicitly teaching for transfer). The study was conducted in general education classrooms, using whole-class instruction and involving all students on whom we had consent. The basic word-problem types targeted for instruction were more complex than had been studied to date with third-grade students, including finding half, step-up functions, two-step problems involving pictographs, and two- and three-step shopping list problems. Third-grade classes were randomly assigned to teacher-designed word-problem instruction, to experimenter-designed instruction on the first three instructional components, or to experimenter-designed schema-broadening instruction that incorporated the first three components but also explicitly taught students to transfer. With the addition of this last instructional component, teachers explained how problem features such as format or vocabulary can make problems seem unfamiliar without modifying the problem type or the required solution rules; they discussed examples emphasizing the same problem type, as superficial features such as format or vocabulary changed; they provided practice in sorting novel problems in terms of superficial problem feature changes and in solving those problems; and they prompted students to search novel problems for familiar problem types. Results indicated that schema-broadening instruction (i.e., all four components) strengthened problem-solving performance over and beyond experimenter-designed instruction that only addressed the first three components (understanding the underlying mathematical structure, recognizing the basic schema, and solving problems) and beyond teacher-designed instruction. Importantly, effects were revealed on a far-transfer performance assessment that required students to solve multiple taught and untaught problem types within a highly novel and complex context that resembled real-life problem-solving situations. Subsequent work (L. S. Fuchs, Fuchs, Finelli, et al., 2004) has shown that schema-broadening instruction that addresses transfer instruction on superficial features is more effective than schema-broadening instruction that addresses transfer instruction on three superficial features. In this series of studies, effect sizes favoring schema-broadening instruction have been large (0.89–2.14). Random assignment, however, has occurred at the general education classroom level (analyses were conducted using classroom as the unit of analysis with low, average, and high performance treated as a within-classroom variable). This level of assignment resulted in relatively small numbers of students at risk for serious math word problem difficulty due to concurrent low performance in math and reading.

We teach students to understand the underlying mathematical structure of the problem type, to recognize the basic schema for the problem type, to solve the problem type, and then to understand how novel problems fit within those schema and can be solved using the same solution strategies.

PURPOSE OF AND INSTRUCTIONAL APPROACH USED IN THE PRESENT STUDY

The main purpose of the present study was to assess the efficacy of a secondary preventative tutoring protocol addressing math word problems at third grade. In addition, we extended research on secondary tutoring protocols generally and our research program on schema-broadening instruction specifically in three ways. First, we screened large numbers of students to identify a subset with documented math and reading difficulties. With difficulty in math and reading, the sample could be expected to have severe deficits in both areas, mirroring the type of students identified in the schools as having risk for or with learning disabilities. In fact, the average reading and math performance of the participants in the study approximated the 10th percentile. Second, at the individual student (rather than classroom) level, we randomly assigned students to receive tutoring or to serve in a control group. By using a control group that continued in their school-designed classroom (Tier 1) math program without tutoring, we could estimate the effects of secondary preventative tutoring (which is designed to be additive to the classroom math program). Third, our schema-broadening tutoring protocol addressed and our dependent measures assessed the major problem types third-grade children are expected to solve: one-step problems requiring adding and subtraction of single- and double-digit numerals, with and without irrelevant information, with and without relevant information embedded in charts, graphs, and pictures, and with missing information in all three positions within the algebraic equation associated with the problem type. We identified these problem types by analyzing a sample of norm-referenced math tests used to identify learning disabilities and for statewide high-stakes accountability testing (Seethaler & Fuchs, 2005).

The present study differs from Jitendra et al.’s studies (2002, 2007) at third grade in three ways. First, our population was students with documented math and reading difficulties. Second, the tutoring protocol relied on schema-broadening instruction (i.e., teaching students to (a) understand the underlying mathematical structure of the problem type; (b) recognize the basic schema for a problem type; (c) solve the problem type; and (d) transfer so they recognize problems with novel superficial features as belonging to the problem type, or schema, for which they know a problem solution). By contrast, Jitendra et al.’s work focuses on the first three instructional components, without explicit instruction to transfer. Third, the present study focused on a broader range of problems.

Before describing our methods and results, we note that because we targeted a population with severe underachievement, we supplemented word problem instruction by addressing the foundational skills required for successful problem solution. Although these foundational skills received less attention than word problems, tutoring was designed to address the variety of skills required for successful word-problem performance. Such a focus on foundational skills to support math problem solving seems necessary within secondary preventative tutoring.

METHOD

DESIGN

We used a randomized controlled trial to estimate the effects of secondary prevention, using an explicit schema-broadening tutoring protocol, on math problem solving at third grade, when contrasted to sole reliance on general education math instruction. We focused on students who began third grade with low performance in math and reading because, given the correlation between math and reading, deficits in both areas are likely to produce severe academic deficits that are especially difficult to remediate. In September and October of the 2005 to 2006 school year, we conducted screening to identify students with difficulty in reading and math. During the last week in November and the first week in December, we pretested students and randomly assigned them to receive tutoring or to remain in the Tier 1 general education program without tutoring. Tutoring lasted for 12 weeks (December 5 through March 14). Posttesting occurred from March 15 through March 17. We note that the sample size is small, permitting the detection of statistically significant effects only for large effect sizes (i.e., > .65). Given that the study is underpowered, we present and discuss effect sizes for nonsignificant effects to reduce the possibility of rejecting a potentially useful intervention, but we caution readers that effect sizes associated with nonsignificant differences should be interpreted with caution.

PARTICIPANTS

To identify a sample of 42 students with mathematics and reading difficulties, we screened 511 third-grade students in 29 classrooms in eight schools in a southeastern urban district, using the following procedure. First, to every student on whom we had obtained consent, we administered the Wide Range Achievement Test 3 (WRAT) Arithmetic (Wilkinson, 1993) subtest in a group setting. To all students scoring below the 26th percentile, we administered WRAT Reading (Wilkinson, 1993) and the two-subtest Weschler Abbreviated Scale of Intelligence (WASI; Psychological Corporation, 1999). The first 42 students who scored below the 26th percentile on WRAT Reading and who achieved a T score exceeding 36 on at least one subtest of WASI were entered into the study. We employed this WASI criterion for study entry because we wanted to exclude students who would potentially qualify as having mental retardation, but we also wanted to permit scatter across cognitive abilities.

Students were randomly assigned to receive schema-broadening tutoring on word problems (tutor) or to continue with their mathematics instruction unchanged (control). Between pre- and posttesting, 5 tutored students and 2 control students moved. Analyses on pretreatment demographic and performance data revealed no significant differences between the students who moved, as a function of treatment condition, and showed that the original sample did not differ significantly from the sample that excluded the students who moved. Demographic and screening data on the remaining 35 students, as a function of treatment condition, are shown in Table 1. As indicated with chi-square analyses, the groups were comparable on sex, race, subsidized lunch status, years in special education, disability status, and English language learner status. As revealed with one-way analyses of variance (ANOVAs), the groups were comparable on WRAT Arithmetic, WRAT Reading, WASI Vocabulary, WASI Matrix Reasoning, and WASI IQ.

TABLE 1.

Demographic and Screening Data by Treatment Condition

	Treatment Condition
	Control (n = 19)			Tutor (n = 16)
Variable	% (n)	M	(SD)	% (n)	M	(SD)	χ2	p	F	p
Demographics
Male	36.8			50.0			0.61	.433
	(7)			(8)
Race
African American	63.2			50.0			1.25	.741
	(12)			(8)
Caucasian	15.8			31.3
	(3)			(5)
Hispanic	15.8			12.5
	(3)			(2)
Biracial	5.3			6.3
	(1)			(1)
Subsidized lunch	84.2			93.8
	(16)			(15)			0.78	.377
Years special education
0	78.9			75.0
	(15)			(12)			2.09	.554
1	5.3			0.0
	(1)			(0)
2	0.0			6.3
	(0)			(1)
3	15.8			18.8
	(3)			(3)			0.61	.736
Disability
Learning	5.3			12.5
	(1)			(2)
Speech/language	15.8			12.5
	(3)			(2)
English language learner	10.5			6.3			0.20	.653
	(2)			(1)
WRAT-Arithmetic		81.95	(5.74)		81.37	(8.46)			0.11	.733
WRAT-Reading		81.37	(8.46)		80.88	(7.18)			0.03	.855
WASI-Vocabulary		38.05	(7.32)		39.75	(11.05)			0.30	.591
WASI-Matrix Reasoning		42.74	(11.82)		43.31	(9.76)			0.02	.878
WASI-IQ		85.32	(8.93)		87.44	(12.40)			0.35	.561

Note. WRAT and WASI are age-based scale scores with a mean of 100 and standard deviation of 15.

WASI-Vocabulary and WASI-Matrix Reasoning are age-based scale scores with a mean of 50 and standard deviation of 10.

TREATMENT

Tier 1 General Education (and Control) Instruction

To guide mathematics instruction, classroom teachers used Math Advantage (Burton & Maletsky, 1999), which relies on teacher-guided problem-solution instruction, with variations in problem cover stories. Instruction addressed the same problem types, with one problem type addressed at a time (as was the case with experimental tutoring). A prescribed set of problem–solution rules was taught (e.g., find the numbers to be combined, set up an addition equation to add the numbers together), but there was no attempt to provide practice in sorting problems into problem types or to broaden students’ schemas. Tier 1 general education instruction was explicit and relied on worked examples, guided group practice, independent work with checking, and homework. Multitier prevention systems and RTI models of identification dictate that preventative tutoring be added to, not substituted for, regular classroom Tier 1 instruction. So, we minimized the time students missed from regular classroom Tier 1 math instruction. In this way, we conceptualized the control group as representative of conventional Tier 1 classroom word problem instruction in order to estimate the contribution of the experimental secondary (Tier 2) preventative tutoring protocol. The 35 students were drawn from 18 classrooms. (We did not conduct observations of Tier 1 general education math sessions or attempt to measure fidelity of implementation of the general education math curriculum.)

Preventative Tutoring

For a manual with teaching scripts and materials, contact the first author. These scripts and materials provided a road map for tutors to implement the tutoring procedures, but the scripts were studied not read. Tutors were students at a local university: one doctoral student with teacher licensure and experience; one student with general education teaching experience studying to obtain a master’s degree and special education teacher licensure; and two students enrolled in a master’s degree program in counseling, with no licensure or teaching experience. Each session was conducted one-to-one outside the student’s classroom and lasted 20 to 30 min. Sessions occurred three times per week for 12 weeks.

Tutoring began with a 2-week introductory unit that taught students foundational skills for successful problem solving, given the tutoring methods. These included (a) adding and subtracting numbers using a number line; (b) solving two-digit adding and subtracting problems with and without regrouping; (c) solving algebraic equations (x + y = z; x − y = z) with missing information in any of the three positions; and (d) applying general mathematics problem-solving strategies (making sure answers make sense; lining up numbers from text to perform math operations; checking computation; and labeling work with words, monetary signs, and mathematical symbols—see Figure 1 for Checking Your Work Poster; these posters reflect the problem–solution rules taught to students). Next, three 3-week units each covered one problem type, with cumulative review of previously taught problem types built into the daily lessons. Then, a final 1-week review unit addressed all three problem types. In each word problem unit session, four activities occurred.

FIGURE 1.

Checking Your Work Posters

The three word problem types, originally conceptualized into a framework by Carpenter and Moser (1984); Riley and Greeno (1988); and Riley, Greeno, and Heller (1983), represent the types of word problems most typically found in mathematics curricula at Grades 1 to 3. Total problems combine two or more quantities to make a total. An example of a total problem with missing information in the third position of the algebraic equation is as follows: There are 51 boys and 47 girls in the third grade at Baker Elementary School. How many third graders are there altogether? Difference problems compare a bigger and smaller quantity to find the difference. An example of a difference problem with missing information in the third position of the algebraic equation is as follows: At the picnic, the kids ate 54 hot dogs. They ate 32 hamburgers. How many more hot dogs did they eat than hamburgers? Change problems involve increasing or decreasing a starting quantity to end up with a new quantity. The following is an example of a change problem with missing information in the third position of the algebraic equation: Jamarius baked 48 cookies. Then, he gave 23 of them to his friends. How many cookies does Jamarius still have? In addition, within each problem type, we explicitly taught students to solve for missing information in the first, second, or third position. Further, we explicitly taught students to transfer to problems that incorporate irrelevant information or that communicated relevant and irrelevant information in charts, graphs, or pictures. The problems also varied the number of digits in the operands (one vs. two digits; see Table 2 for a listing of problems from the Peabody Word Problems Test, L. S. Fuchs & Seethaler, 2005), showing problem type, position of missing information, and whether the problem involves irrelevant numbers or included charts and graphs. This measure incorporates two-digit numbers; Jordan’s Story Problems Test, which addresses more straightforward problems, includes only one-digit operands.

TABLE 2.

Peabody Word Problem Test

Problem Text		Problem Type	Position of Missing Information	Bar Chart, Pictograph, or Scene
1.	The art teacher had 42 pieces of construction paper. Some of the pieces of paper were blue, and 20 pieces of the paper were green. How many pieces of blue paper did she have?	Total	A
2.	Johnny had 21 pencils in his desk. Then he found another 16 pencils in the closet. How many pencils does Johnny have now?	Change	C
3.	On Monday, Aaron folded some baby blankets for his parents. On Tuesday, his dad gave him 34 more blankets to fold. He folded 59 blankets in all. How many blankets did he fold on Monday?	Change	A
4.	At the picnic, the kids ate 54 hot dogs. They ate 32 hamburgers. How many more hot dogs did they eat than hamburgers?	Difference	C
5.	At school, Dylan used 19 tissues. When he got home, he used some more tissues. He used 49 tissues in all. How many tissues did Dylan use at home?	Change	B
6.	Jamarius baked 48 chocolate chip cookies. Then he gave 23 of them to his friends. How many cookies does Jamarius still have?	Change	C
7.	Ms. Luther’s class checked out 62 books from the library. The students read some of the books and returned them to the library. Now her class has 40 books left to read. How many books did they return to the library?	Change	B
8.	There are 51 boys and 47 girls in the third grade at Baker Elementary School. How many third graders are there altogether?	Total	C
9.	On Sunday, Charles put some flowers in a vase. The next day, 43 died. There are 23 flowers still alive. How many flowers did put in Charles the vase on Sunday?	Change	A
10.	A new t-shirt and watch cost $62 in all. How much does a watch cost?	Total	B	Pictograph
11.	Last year, there were only 11 trumpet players in the band. This year, some more trumpet players joined the band. The picture shows how many trumpet players there are in high school. How many new trumpet players joined the band this year?	Change	B	Scene
12.	Rachel brought some money to the book store, where she spent some of that money on a comic book. Now she has $45 left. How much money did Rachel bring to the book store?	Change	A	Scene
13.	Mr. Ramirez brought $75 to the electronics store. He bought a digital camera and a CD player. How much money did he have left?	Change	C	Bar chart
14.	Treasure Chest D has $66 less than Treasure Chest B. How much money is in Treasure Chest B?	Difference	A	Scene
15.	Third graders at Baker Elementary School wrote stories on the computer. Alex wrote for 45 minutes. The graph shows the number of pages he wrote. When he turned off the computer, he forgot to save his work and 16 pages were deleted. How many pages does he have left of his story?	Change	C	Pictograph
16.	Maria wrote some pages of her story last week. Today in computer class, she wrote another 18 pages. Maria also played MathBlaster for 20 minutes during computer class. The graph shows the total number of pages she has written. How many pages did Maria write last week?	Change	A	Pictograph
17.	Natasha spent $15 on rollerblades at the toy store. Then she spent some money on a jewelry box at the gift shop. How much money did Natasha spend?	Change	C	Bar chart
18.	Bus tickets to Alabama used to cost $18. Alabama is 342 miles away. Then the bus station changed the prices for the tickets. The graph shows the new prices. How much did the price for a ticket to Alabama increase?	Change	B	Bar chart

Basic Facts Flash Cards

The first activity was basic facts flash cards, for which the student was presented with up to 50 addition and subtraction flash cards, including families with sums from 0 to 18. The tutor showed one card at a time; students had 2 min to respond to as many cards as possible, as the tutor sorted responses into correct and incorrect stacks. The student counted and graphed the number of correct responses. The tutor provided corrective feedback on up to five errors and reshuffled the deck of flash cards for the next session.

Underlying Mathematical Structure, Schema Recognition, Problem–Solution Rules, and Transfer Instruction

The second activity, which lasted 12 to 17 min, was the schema-broadening instruction lesson (contact first author for a sample script). Students were taught to apply a three-step process for approaching every problem, regardless of problem type, which we termed the RUN strategy. Students were taught to “run through” the problem by first reading it, then underlining the question, and finally naming the problem type (total, difference, or change; see Figure 1). The RUN strategy applied to every problem type.

Students were taught to “run through” the problem by first reading it, then underlining the question, and finally naming the problem type.

Next, for each problem type, the underlying mathematical structure was taught using concrete materials and role playing. Next, using the algebraic equation templates shown in the posters (see Figure 1), students were taught to recognize problems with the given mathematical structure as a problem that belonged to a given problem type. Third, a set of solution rules was taught. For total problems, students were taught to find Part 1 and label that number P1, to find Part 2 and label that number P2, and to find the total and label that number T. Then, students wrote the number sentence (entering x for the missing piece of information) and solved for x (see Figure 1). For difference problems, students were taught to find the bigger number and label it B, to find the smaller number and label that number S, and to find the difference and label that number D. Then, students wrote the number sentence (entering x for the missing piece of information) and solved for x (see Figure 1). For change problems, students were taught to find the number to start and label it St, to find how many changed and label that number C, to determine whether the start number increased (therefore selecting +) or decreased (therefore selecting −); and to find the number at the end and label that number E. Then, students wrote the number sentence (entering x for the missing piece of information) and solved for x (see Figure 1). We note that the tutoring methods minimized the use of key words in instruction and in the problems used for instruction.

Throughout all the units except the introductory one, students were explicitly taught to transfer in four ways. First, in the second lesson of the total problem-type unit, tutors taught students how irrelevant information can make a novel problem for which they know a problem solution appear unfamiliar. Students were explicitly taught to be aware of, find, and cross out irrelevant information. Second, in the fourth lesson of the total, difference, and change units, tutors taught students about how missing information in the first or second position of the algebraic equation represented in the problem narrative can make a novel problem for which they know a problem solution appear unfamiliar. Students were explicitly taught to be aware of missing information in the first or second position and how to address it while using the solution strategies they had learned. Third, in the seventh lesson of the total, difference, and change units, tutors taught students to apply their word problem skills to problems that required addition and subtraction with two-digit operands. Students were explicitly taught to set up problems using algebraic equations and then to translate those problems to vertical formats for solution. Fourth, after each of the problem types had been taught, in a final review unit, tutors taught students about how charts, graphs, and pictures can make a novel problem for which they know a problem solution appear unfamiliar. Students were explicitly taught to be aware of and use charts, graphs, and pictures within the problem types for which they had learned solutions. After these transfer features (irrelevant information and charts, graphs, and pictures) were initially introduced, they were cumulatively reviewed.

Next, within each session, students did an activity called “Sorting Word Problems.” Students were presented with a random assortment of problems representing the three problem types, which were printed on flash cards. These problems incorporated similar cover story characters but varied actions and numbers to create an assortment of total, difference, and change problems, with missing information in different positions. For 2 min, the tutor read problems, one at a time; the student responded by identifying whether the problem belonged to the total, difference, or change problem type. The student counted and graphed the number of correct responses. The tutor provided corrective feedback on up to five errors and reshuffled the deck of problem cards for the next session.

Daily Review

The final activity in each session was a daily review for which the student had 2 min to complete 10 addition and subtraction problems with single-digit operands and four addition and subtraction problems with double-digit operands. Then, the student had another 2 min to complete one word problem. The tutor corrected errors by modeling the correct responses out loud while the student watched.

Reinforcement

During all activities, students earned tokens for correct responses, which they recorded at the end of the sessions and traded in weekly for prizes. The theme for instruction and reinforcement was “Pirate Math,” which was reflected in scripts and posters. Also, gold coin tokens were recorded each day on the student’s treasure map and traded in weekly for prizes purchased from the treasure chest.

Delivery

All sessions were scripted to ensure consistency of information; however, to permit natural teaching styles, scripts were studied, not read. To ensure comparable mathematics instructional time across conditions, experimental sessions typically occurred within the confines of the mathematics instructional block.

Treatment Fidelity

Every tutoring session was audiotaped. Three research assistants independently listened to tapes while completing a checklist to identify the percentage of essential points in the lesson that had been addressed. We sampled 20.3% of the tapes such that research assistants and lesson types were sampled equitably. The mean percentage of points addressed was 99.72 (SD = 0.01).

MEASURES

We employed measures of the foundational skills involved in problem solving and measures of word problem skill.

Foundational Skills

We measured six types of foundational skills involved in problem solving: fluency with addition and subtraction facts, double-digit addition and subtraction with and without regrouping, skill in solving algebraic equations, and word reading skill.

Addition Fact Fluency (L. S. Fuchs, Hamlett, & Powell, 2003) comprises 25 addition fact problems with answers from 0 to 18, presented horizontally on one page. Students have 1 min to write answers. The score is the number of correct answers. Staff entered responses into a computerized scoring program on an item-by-item basis, with 15% of tests reentered by an independent scorer. Data entry agreement was 100%. Coefficient alpha on this sample was .93. Subtraction Fact Fluency (L. S. Fuchs, Hamlett, et al.) comprises 25 subtraction fact problems with answers from 0 to 18, presented horizontally on one page. Students have 1 min to write answers. The score is the number of correct answers. Staff entered responses into a computerized scoring program on an item-by-item basis, with 15% of tests reentered by an independent scorer. Data entry agreement was 100%. Coefficient alpha on this sample was .81.

The Double-Digit Addition Test (L. S. Fuchs, Hamlett, et al.) provides students 5 min to complete 20 two-digit by two-digit addition problems with and without regrouping. As reported by L. S. Fuchs et al., coefficient alpha was .94; criterion validity with the previous spring’s TerraNova (CTB/McGraw-Hill, 1997) Total Math score was .50. The Double-Digit Subtraction Test (L. S. Fuchs, Hamlett, et al.) provides students 5 min to complete 20 two-digit by two-digit subtraction problems with and without regrouping. The score is the number of correct answers. Coefficient alpha was .92; criterion validity with the previous spring’s TerraNova Total Math score was .46. With WRAT-Arithmetic (Wilkinson, 1993), students have 10 min to complete calculations problems of increasing difficulty. If the basal is not met, they read numerals aloud to the examiner. Median reliability is .94 for ages 5 to 12 years.

With Algebraic Equations (L. S. Fuchs & Seethaler, 2005), students complete eight simple addition and subtraction algebraic equations that include missing information in all three positions (e.g., 2 + 3 = x; x + 3 = 5; 2 + x = 5). The score is the number of correct answers. Coefficient alpha was .84; criterion validity with the previous spring’s TerraNova Total Math score was .46.

With WRAT-Reading (Wilkinson, 1993), students read aloud letters and words until a ceiling is reached. Reliability is .94.

Word Problems

These measures all sampled novel problems (i.e., never seen before or used for instruction). Story Problems (Jordan & Hanich, 2000, adapted from Carpenter & Moser, 1984; Riley & Greeno, 1988; Riley et al., 1983) comprises 14 brief story problems involving sums or operands of 9 or less, with total, difference, and change problem types, with missing information in all three positions. There are no irrelevant numbers, chart, or graphs. The tester reads each item aloud; students have 30 s to respond. Coefficient alpha on this sample was .86.

With the Peabody Word Problem Test (L. S. Fuchs, Seethaler, & Hamlett, 2005), students complete 18 problems (see Table 2) that include total, difference, and change problem types with missing information in all three positions, with and without irrelevant information, and with and without charts and graphs. The tester reads each item aloud; students have 1 min to respond. Coefficient alpha on this sample was .86.

KeyMath-Revised Problem Solving (Connolly, 1998) includes 18 items of increasing difficulty distributed across three domains: solving simple word problems involving all four operations, comprehending salient features of nonroutine problems, and solving multistep, nonroutine problems. Items are read aloud to the student individually, and testing is discontinued after three consecutive errors. Split-half reliability for Grade 3 is .72. Correlations with the Total Mathematics score of the Comprehensive Test of Basic Skills (CTB/McGraw-Hill, 1981) and the Iowa Test of Basic Skills (Hoover, Hieronymous, Dunbar, & Frisbie, 1993) for Grades 1 through 8 averaged .60.

With the Iowa Test of Basic Skills: Problem Solving and Data Interpretation (Hoover et al., 1993), students solve brief word problems and use data presented in tables and graphs to solve word problems, with 24 to 30 items depending on level. The tester reads each item aloud; students had 30 s to respond. At Grades 1 through 5, KR20 (a form of reliability) is .83 to .87.

DATA COLLECTION

Trained research assistants collected data in small groups. Pretesting occurred during the week before treatment began; posttesting occurred during the week after treatment ended. Every story problem was read aloud, with research assistants providing students time to complete work before progressing to the next item.

DATA ANALYSIS AND RESULTS

We applied one-way ANOVA, using condition as the between-subjects variable, to (a) pretreatment scores to examine treatment group comparability and (b) improvement scores to investigate tutoring efficacy. Given that we included a large number of measures for some constructs, we controlled for Type I error by adjusting p values as follows. For computation measures, for which we had five measures, we adjusted the critical p value to .010 (.05 divided by 5). For algebraic equations, for which we had only one measure, we did not adjust the alpha level, and the same was true for reading, for which we had only one measure. For word problems, for which we had four measures, we adjusted the p value to .0125 (.05 divided by 4). To compute effect sizes (ESs) on pretreatment scores, we subtracted the difference between the pretreatment means and then divided by a pooled standard deviation. To compute ESs on improvement scores, we subtracted the difference between improvement means and then divided by the pooled standard deviation of the improvement/square root of 2(1 − r_xy; Glass, McGaw, & Smith, 1981).

Table 3 shows means and standard deviations for each measure, along with ESs and F values for pretreatment and improvement scores. Where p values were below the critical level, we provide the p value in parentheses. Groups were comparable prior to the study, as manifested by the lack of significant main effects for condition on all pretreatment measures. On most of the foundational skills involved in solving word problems (i.e., addition and subtraction fact retrieval, double-digit addition and subtraction, simple algebraic equations, and WRAT Reading), improvement at statistically significant levels did not vary as a function of treatment condition (i.e., control vs. tutoring). However, ESs were moderate on subtraction fact retrieval, double-digit addition, and simple algebraic equations. Moreover, a significant and practically important effect did emerge on WRAT Arithmetic. On word problems, effects were large and significant on Jordan’s Story Problems (assessing simple problems with one-digit operands and no irrelevant information or charts, graphs, or pictures) and on Peabody Word Problems (assessing more complex problems with two-digit operands, with and without irrelevant information or charts, graphs, or pictures). Although results on the commercial tests of word problems were not statistically significant, the ES on KeyMath Problem Solving was moderate.

TABLE 3.

Means and Standard Deviations by Treatment Condition With F Values and Effect Sizes

	Control (n = 19)		Tutor (n = 16)
Variable	M	(SD)	M	(SD)	F(1,33)	ES
Component Skills
Addition fact retrieval
Pre	9.58	(4.14)	11.81	(3.12)	3.14	0.61
Post	11.32	(4.88)	14.56	(4.00)
Improve	1.74	(3.65)	2.75	(3.82)	0.64	0.23
Subtraction fact retrieval
Pre	5.16	(3.18)	5.25	(2.84)	0.08	0.09
Post	7.74	(4.51)	9.69	(3.68)
Improve	2.58	(4.10)	4.44	(3.01)	2.26	0.51
Double digit addition
Pre	14.21	(6.34)	14.38	(5.43)	0.07	0.03
Post	14.58	(5.75)	17.56	(4.13)
Improve	0.37	(7.00)	3.19	(4.05)	2.02	0.53
Double digit subtraction
Pre	5.53	(3.27)	7.25	(4.47)	1.73	0.45
Post	6.11	(4.81)	9.81	(6.57)
Improve	0.58	(5.06)	2.56	(5.24)	1.29	0.38
WRAT 3 Arithmetic
Pre	21.58	(1.92)	22.00	(1.55)	0.50	0.24
Post	22.89	(2.26)	26.25	(2.75)
Improve	1.32	(2.24)	4.25	(2.35)	14.26 (p = .001)	1.34
Simple algebraic equations
Pre	3.21	(3.05)	4.44	(2.66)	1.58	0.43
Post	3.16	(3.34)	6.06	(1.81)
Improve	−0.05	(4.20)	1.63	(2.58)	1.15	0.58
WRAT 3 Reading
Pre	24.53	(3.08)	24.75	(3.32)	0.04	0.07
Post	26.05	(2.88)	26.88	(2.83)
Improve	1.53	(1.61)	2.13	(2.03)	0.95	0.20
Word problems
Jordan’s Story Problems
Pre	8.37	(3.70)	8.56	(3.22)	0.03	0.05
Post	7.37	(4.25)	9.83	(2.80)
Improve	−1.00	(2.48)	1.27	(2.02)	7.55 (p = .010)	0.69
Peabody Word Problems
Pre	4.00	(2.94)	4.38	(2.78)	0.15	0.13
Post	5.37	(3.11)	12.88	(6.76)
Improve	1.37	(2.99)	8.50	(6.16)	19.96 (p < .001)	1.80
KeyMath problem solve
Pre	2.79	(1.90)	3.50	(1.59)	1.40	0.40
Post	3.53	(1.84)	5.06	(1.48)
Improve	0.74	(1.94)	1.56	(1.46)	1.96	0.47
ITBS data interpretation and problem solve
Pre	9.47	(3.03)	10.13	(3.90)	0.31	0.19
Post	10.37	(4.11)	11.81	(4.14)
Improve	0.89	(2.87)	1.69	(3.93)	0.48	0.21

Note. ITBS = Iowa Test of Basic Skills.

DISCUSSION

In the context of this randomized field trial, third-grade students who received 15 hr of explicit schema-broadening preventative tutoring, distributed over 12 weeks, improved significantly more than comparable peers who remained in the Tier 1 general education program without tutoring. Significant effects were revealed on the primary measures of word problem skill: on Jordan’s Story Problems, assessing simple problems with one-digit operands and no irrelevant information or charts, graphs, or pictures, as well as on Peabody Word Problems, which assess problems with two-digit operands and that include irrelevant information or involve using information derived from charts, graphs, or pictures. The ESs on these measures were sizeable: 0.69 and 1.80 standard deviations, respectively. Result echo findings at first grade (L. S. Fuchs, Compton et al., 2005), showing that secondary preventative tutoring in mathematics can substantially enhance performance on concepts, computation, and word problems. Present findings extend those findings to third grade, where the present study focused specifically on mathematical problem solving. In this way, the present study documents the efficacy a third-grade secondary preventative tutoring protocol that teaches students to (a) focus on the mathematical structure of three problem types; (b) recognize problems as belonging to those three problem-type schemas; (c) solve the three word problem types; and (d) transfer their problem-solving skills to problems that include irrelevant information, two-digit operands, missing information in the first or second positions in the algebraic equation, or relevant information in charts, graphs, or pictures. In addition, the tutoring protocol taught students the calculation and algebraic skills foundational for problem solving.

Of course, present findings cannot comment on whether schema-broadening tutoring is more effective than competing forms of secondary preventative tutoring. Other studies that contrast alternative tutoring protocols must be conducted to answer that question. Present findings do nevertheless suggest the potential usefulness of schema-based strategy instruction beyond students with specific math difficulty (e.g., Jitendra et al., 1998) to students with substantial deficits in math as well as reading. Findings also suggest that our explicit schema-broadening instruction may strengthen mathematical problem solving beyond representative populations of third graders (e.g., L. S. Fuchs, Fuchs, et al., 2003, L. S. Fuchs, Fuchs, Prentice, et al., 2004), again to students with substantial math and reading deficits. The possibility that schema-broadening instruction may be useful for students with substantial math and reading deficits is notable given that students with learning disabilities, who typically have similarly low reading and math skills, are at particular risk for difficulty with word problems (Compton, Fuchs, & Fuchs, 2006; L.S. Fuchs & Fuchs, 2002; Parmar, Cawley, & Frazita, 1996). Future research should continue to focus on this tutoring protocol so that specific effects can be pinpointed.

At the same time, on the commercial tests of word problem skill, effects failed to achieve statistical significance. Given the small sample size and the risk of Type II error (i.e., failing to document that true differences exist due to inadequate statistical power), we discuss ESs associated with nonsignificant differences. In a small sample size study such as the present investigation, these ESs are informative for planning future research. We do, however, caution readers that ESs associated with nonsignificant differences may be due to chance and do not provide the basis for strong conclusions. On KeyMath Problem Solving, the ES was a moderate 0.47, indicating that a larger experimental field trial, with appropriate power, may yield statistically significant differences. This is notable given that the KeyMath is often used to identify students with math disabilities. In a similar way, on the Iowa Test of Basic Skills subtest, the ES was 0.21 standard deviations, again suggesting the need for larger experimental investigations. Although an ES of 0.21 meets the criterion for efficacy used in the What Works Clearinghouse (see www.whatworks.ed.gov), it is disappointingly low given that some version of the Iowa Test is used in many high-stakes state assessment programs (and is similar to some other widely used state assessments). Moreover, we had designed the Peabody Word Problems Test, for which effects were reliable and strong, to reflect the kinds of items found on tests like the Iowa. The major difference between the Iowa and the other word problem measures in this study was the multiple-choice format, suggesting the need to explicitly address this form of transfer in the schema-broadening tutoring.

It is also interesting to consider effects on foundational skills required for competent word problem skill, given the nature of our secondary preventative tutoring. The ESs on the measures of foundational skills varied. For fact retrieval, ESs were 0.23 for addition and 0.51 for subtraction. For two-digit procedural addition and subtraction, the ESs were similar but reversed in magnitude: 0.53 for addition and 0.38 for subtraction. More impressively, on WRAT, a test often used in schools to identify learning disabilities, effects were statistically significant, with a large ES of 1.34 standard deviations. Given the relatively limited time allocated to fact retrieval and procedural calculation within the tutoring, these findings are notable and suggest that work on problem solving, which incorporates practice on these component skills, may result in improved computational skill. Again, larger experimentation is needed to investigate this possibility.

In addition to indexing fact retrieval and procedural computation, we also measured students’ skill with algebraic equations, which was a major strategy we used to teach word problem skill. The effect, although not statistically significant, was a moderate 0.58, suggesting that the tutoring may be efficacious in promoting this outcome. This is notable in that the students were not only severely deficient in incoming math skill but also third graders. Given the strong focus on algebra in high schools and the requirement in many states that students pass an algebra course or an algebra test prior to graduation, introducing algebra this early in the curriculum may represent an important and productive innovation. As with the schema-broadening problem-solving intervention for students with reading and math difficulties, such a conclusion awaits larger experimentation, which is now underway.

EDUCATIONAL IMPLICATIONS

Pending additional research, several implications are relevant to the practice of special education. First, the tutoring protocol validated in the present study should be useful to school practitioners as they implement responsiveness-to-intervention models of learning disabilities identification at third grade. Results show that this schema-broadening protocol, when used as a secondary preventative tutoring program and when implemented with fidelity, should be effective in improving word problem performance. Second, findings suggest the potential efficacy of introducing algebraic equations to support math problem solving as early as third grade and with students who demonstrate severe difficulties in reading and mathematics.