Contribution of Equal-Sign Instruction beyond Word-Problem Tutoring for Third-Grade Students with Mathematics Difficulty

Abstract

Elementary school students often misinterpret the equal sign (=) as an operational rather than a relational symbol. Such misunderstanding is problematic because solving equations with missing numbers may be important for higher-order mathematics skills including word problems. Research indicates equal-sign instruction can alter how typically-developing students use the equal sign, but no study has examined effects for students with mathematics difficulty (MD) or how equal-sign instruction contributes to word-problem skill for students with or without MD. The present study assessed the efficacy of equal-sign instruction within word-problem tutoring. Third-grade students with MD (n = 80) were assigned to word-problem tutoring, word-problem tutoring plus equal-sign instruction (combined) tutoring, or no-tutoring control. Combined tutoring produced better improvement on equal sign tasks and open equations compared to the other 2 conditions. On certain forms of word problems, combined tutoring but not word-problem tutoring alone produced better improvement than control. When compared at posttest to 3^rd-grade students without MD on equal sign tasks and open equations, only combined tutoring students with MD performed comparably.

Elementary school students often understand the equal sign (=) as an operational symbol when it should be viewed as a relational symbol (Ginsburg, 1977). This misinterpretation may lead to difficulty in solving word problems and equations (Lindvall & Ibarra, 1980; McNeil & Alibali, 2005). A connection between equal-sign understanding and word-problem solving may exist because students often spontaneously or are taught to generate and solve a number sentence to represent information in problem narratives (e.g., Carpenter, Moser, & Bebout, 1988; Fuchs, Seethaler et al., 2008). Equal-sign instruction may alter how typically-developing students understand and use the equal sign (e.g., Rittle-Johnson & Alibali, 1999), but no study has examined effects for students with mathematics difficulty (MD) or how equal-sign instruction contributes to word-problem skill for students with or without MD. The purpose of the present study was to assess the contribution of equal-sign instruction beyond word-problem tutoring on equal-sign understanding and word-problem skill of third-grade students with MD.

In this introduction, we explain why word problems represent a challenging domain for students with MD and summarize prior work on the efficacy of schema-broadening word-problem instruction, which incorporates equations to represent the underlying structure of problems. We then transition to understanding the equal sign, explaining its relevance to word problems, and summarizing previous research on instruction designed to promote a relational understanding of equation solving. Finally, we propose the hypotheses and causal mechanism for the present study, which addresses the potential connections among equal-sign understanding, equation solving, and word-problem solving.

Before proceeding, we comment on terminology. In the literature, mathematics disability is operationalized as low mathematics performance and referred to as mathematics difficulty. In this paper, we do the same. We focused the present study on students with MD because these students generally struggle across the various domains of mathematics and because, as we illustrate, perform significantly and dramatically lower than typically-developing peers on tests assessing equal-sign knowledge, equation solving, and word-problem solving. Within the MD category, we recruited students with MD and concomitant reading difficulty (MDRD) or MD alone (MD-only). MDRD and MD-only students have previously been conceptualized as constituting two distinct groups (e.g., Geary, Hamson, & Hoard, 2000). We deemed this categorization as potentially salient to the present study because the language profiles of students with MDRD are generally weaker compared to students with MD-only and because word-problem solving relies on language abilities (Fuchs, Fuchs et al., 2008). Also, throughout the paper, we refer to equations as standard or nonstandard and closed or open. Standard equations are in the form of number, operator symbol, number, equal sign, number (e.g., 2 + 4 = __; 2 + 4 = 6); they can be open (i.e., incorporating a blank or variable to solve) or closed (without any missing information). Nonstandard equations occur in any form other than standard (e.g., 6 + 4 = __ + 8; 6 = 2 + 4) and can also be open or closed. (See Table 1 for a reference guide and other examples; our categorization is similar to equation terminology in most prior work.)

Table 1. Equation Terminology.

Type of equation	Definition	Examples
Standard	An equation in the form of number, operator symbol, number, equal sign, number	2 + 4 = 6	8 + __ = 11
		12 - 5 = 7	__ - 3 = 2
Nonstandard	An equation not in the form of number, operator symbol, number, equal sign, number	6 = 2 + 4	11 = __ + 8
		7 = 12 - 5	2 = __ - 3
		3 + 2 = 4 + 1	8 - __ = 9 - 3
		10 - 7 = 6 - 3	6 + 4 = __ + 8
Closed	An equation without missing information	10 - 7 = 3	14 - 6 = 12 - 4
		8 = 4 + 4	3 + 4 = 1 + 6
Open	An equation with missing information (i.e., a blank or variable to solve)	10 - __ = 3	14 - 6 = 12 - __
		8 = 4 + __	3 + 4 = __ + 6
		2 + 2 = X	5 - 2 = X - 4
		? - 5 = 8	3 + n = 8 - 6

Note: Some of this terminology is similar to other researchers; some is not. All equations are a combination of standard or nonstandard and closed or open. The four possible equation combinations are standard/closed, standard/open, nonstandard/closed, and nonstandard/open.

Word-Problem Solving

A significant challenge in the elementary grades, which may be connected to students' understanding of the equal sign, is word problems. This is especially true for students with MD (Fuchs & Fuchs, 2002). Difficulty can arise because various steps and skills are needed to solve word problems (Parmar, Cawley, & Frazita, 1996) or because embedding mathematics within a linguistic context challenges students who also may experience language deficits (Fuchs, Fuchs et al., 2008). To solve a word problem, students use the problem narrative to develop a problem model and identify the missing information, generate a number sentence that represents the problem model and incorporates the missing information, and derive the calculation problem for finding the missing information. Generating a number sentence to represent a word problem, an important step for solving a word problem correctly (Carpenter et al., 1988), is difficult (Herscovics & Kieran, 1980). Moreover, even when students have generated a number sentence to represent a word problem, solving the number sentence can represent a substantial hurdle, especially if students misunderstand the equal sign and cannot solve equations correctly.

To help students with MD become better at word problems, explicit word-problem instruction has been proven effective (Kroesbergen, Van Luit, & Maas, 2004). Some recent work has focused on the development of schemas, with which students are taught to recognize problems as belonging within problem types and to apply solution strategies that match those schemas (e.g., Fuchs, Fuchs, Finelli, Courey, & Hamlett, 2004; Jitendra, Griffin, Deatline-Buchman, & Sczesniak, 2007). Developing schemas for categorizing word problems is beneficial because it helps students understand novel problems as belonging to familiar categories (Cooper & Sweller, 1987) and helps address MD students' working memory deficits, which have been linked to difficulty with word problems (e.g., Swanson & Beebe-Frankenberger, 2004). With schema-broadening instruction (e.g., Fuchs et al., 2009; Fuchs, Seethaler et al., 2008), students classify word problems in terms of schemas (from the problem types presented during instruction) and then apply a solution strategy that matches that schema.

Two schema-broadening experiments have taught students to represent the underlying structure of problem types using algebraic equations. Fuchs, Seethaler et al. (2008) randomly assigned third-grade students with MDRD to schema-broadening tutoring or no-tutoring control. Schema-broadening tutoring occurred for 12 weeks, one-to-one, with three 30-min sessions per week. Instruction focused on three problem types. For each problem type, students learned to understand the schema, to represent the schema with an algebraic equation, and to solve equations. Then, students were taught to recognize problems with novel features as belonging to the three schemas. Students receiving this schema-broadening tutoring grew significantly better than control students on word problems, with effect sizes (ESs) ranging from 0.69 to 1.80.

Fuchs et al. (2009) expanded the earlier study to control for tutoring time with a contrasting tutoring condition. Third graders with MD at two sites were randomly assigned to three conditions: number combinations tutoring, schema-broadening word-problem tutoring with algebraic equations, and no-tutoring control. Individual tutoring occurred 3 times per week for 15 weeks, with 20-30 min sessions. On word problems, students who received schema-broadening word-problem tutoring with algebraic equations significantly outperformed students in number-combination tutoring and in the control group (ESs = 0.83 and 0.79, respectively). These studies reveal how students with MD benefit from tutoring that incorporates algebraic equations within schema-broadening instruction.

Two Understandings of the Equal Sign

Given the focus on representing problems with algebraic equations and solving equations in schema-broadening instruction, understanding of the equal sign may be important. The equal sign should be understood as a relational symbol, indicating that a balanced relationship exists between numbers on the two sides of the equivalence symbol (Jacobs, Franke, Carpenter, Levi, & Battey, 2007). Young students demonstrate an understanding of equivalence by counting two sets and stating whether the sets are the same (Kieran, 1981). Unfortunately, students come to misunderstand the equal sign as they implicitly develop ideas about addition and subtraction before entering school (Seo & Ginsburg, 2003) and as they experience early elementary school instruction that exclusively presents equations in standard form (e.g., 2 + 3 = __; Capraro, Ding, Matteson, Capraro, & Li, 2007). Such a heavy focus on standard equations in textbooks and school work leads students to an operational understanding of the equal sign, which signals them to do something (Saenz-Ludlow & Walgamuth, 1998) or find the total (McNeil & Alibali, 2005).

Correct understanding of the equal sign is important due to its potential role in higher-level mathematics, including algebra and word problems. In terms of algebra, a relational understanding of the equal sign helps students handle open standard equations (e.g., 6 + __ = 10) and open or closed nonstandard equations (e.g., 5 = 9 − 4 or 3 + 5 = __ + 4). When students believe the equal sign means to perform an operation, they often view closed equations as incorrect (e.g., Carpenter & Levi, 2000) or solve open nonstandard equations incorrectly (e.g., Lindvall & Ibarra, 1980; Weaver, 1973). In fact, students who interpret the equal sign in a relational manner more successfully solve algebraic equations (Knuth, Stephens, McNeil, & Alibali, 2006). With respect to word problems, many students spontaneously generate or are taught to represent the underlying structure of word problems with algebraic equations (Carpenter et al., 1988). If an operational understanding of the equal sign compromises understanding of equations as well as solution accuracy, then word-problem performance will also suffer (Carpenter, Franke, & Levi, 2003).

Strategies for Promoting a Relational Understanding of the Equal Sign

The question therefore is how to promote a relational understanding of the equal sign. Non-experimental studies (Baroody & Ginsburg, 1983; Blanton & Kaput, 2005; Saenz-Ludlow & Walgamuth, 1998) suggest dialogue may be effective. Researchers have also conducted experiments to increase insight into the benefits of equal-sign instruction. McNeil and Alibali (2005) randomly assigned students ages 7 through 11 to problem structure intervention, equal-sign intervention, and control conditions. All students were presented with a correctly solved equation (6 + 4 + 7 = 6 + 11). Control students thought about the equation for 1 min. The problem structure condition focused students on the location of the equal sign. The equal-sign condition focused students on the meaning of the equal sign and the idea that both sides of the equation must be equal. At pre- and posttest, students worked with open nonstandard equations (e.g., 3 + 4 + 5 = __ + 5) and defined the equal sign. Before intervention, 87% of students added up all the numbers, and none defined the equal sign in a relational manner. At posttest, students in the problem structure intervention were able to reconstruct an open nonstandard equation, and students in the equal-sign intervention had better understanding of the equal sign as a relational symbol.

Focusing on similar equations, Rittle-Johnson and Alibali (1999) assessed the effects of explicit equivalence instruction. Fourth and fifth graders who missed problems on an equivalence screening measure were randomly assigned to conceptual intervention, procedural intervention, and control conditions and were pretested on conceptual and procedural knowledge of equivalence using open nonstandard equations (e.g., 3 + 4 + 5 = 3 + __). In the same session, conceptual students were presented with an equivalence problem and told the principle (i.e, numbers on each side of the equal sign need to be equal). Procedural students received instruction on how to solve for the missing information. Control students received no instruction. On the next day's posttests, students who received instruction significantly outperformed controls on conceptual and procedural tests. Although raw scores favored students in the conceptual over the procedural treatment, there were no significant differences.

Hypotheses and Proposed Causal Mechanism for the Present Study

Prior work on schema-broadening instruction and the relational meaning of the equal sign provides the basis for several related hypotheses. First, students, especially those in the early elementary grades, require better instruction on the relational meaning of the equal sign to avoid or correct early misunderstanding of the language of mathematics (Falkner, Levi, & Carpenter, 1999), and explicit instruction holds promise (McNeil & Alibali, 2005; Rittle-Johnson & Alibali, 1999). We therefore hypothesized that, on tasks assessing equal-sign understanding, combined tutoring (word-problem combined with equal-sign instruction) would produce significantly better outcomes than word-problem tutoring and control, which would perform comparably to each other (COMB > WP = CON).

In terms of solving open equations, we had two hypotheses. First, because combined tutoring as well as word-problem tutoring involved instruction on setting up and solving standard algebraic equations, we hypothesized that both tutoring conditions would significantly outperform control on solving standard equations, and the two active tutoring conditions would perform comparably (COMB = WP > CON). Second, with the addition of equal-sign instruction, we predicted that on unfamiliar (i.e., nonstandard) equations combined tutoring students would outperform word-problem tutoring and control students, which would perform comparably to each other (COMB > WP = CON).

Finally, research indicates that students with MD benefit from explicit schema-broadening word-problem instruction that incorporates algebraic equations to represent the underlying structure (i.e., schema) of word problems (e.g., Fuchs et al., 2009). Because schema-broadening instruction was provided to combined and word-problem tutoring students alike, we hypothesized that both active tutoring conditions would perform comparably on word problems, with both active tutoring conditions outperforming control on word problems with missing information after the equal sign (COMB = WP > CON). Yet, on word problems with missing information before the equal sign, we predicted that equal-sign tutoring would provide a differential boost to combined tutoring students over word-problem and control students (COMB > WP = CON).

Across all five hypotheses, we were also interested in the performance differences of MDRD and MD-only students. Because MDRD students struggle with reading text and reading is an integral part of solving word problems and because word problems involve language abilities, we projected that MD-only students would respond better than MDRD students to word-problem tutoring.

We connected our hypotheses to predict that instruction on the relational meaning of the equal sign with a focus on solving equations, when combined with schema-broadening instruction that incorporates algebraic equations to represent the underlying structure of word problems, would provide added value over and above schema-broadening word-problem instruction that does not also address the relational meaning of the equal sign. We based this prediction on the following proposed causal mechanism. Students provided with explicit equal-sign instruction will learn to interpret the equal sign in a relational manner. Understanding the equal sign relationally will transfer to skill at solving open equations. When students understand the equal sign relationally and are more effective at solving open equations, they will better understand how algebraic equations represent word-problem schema and therefore they will generate equations more accurately, even as they will solve the algebraic equations representing the word problems more efficiently. Better understanding of how algebraic equations represent word-problem schema and superior algebraic-equation solving will, in these ways, promote superior word-problem skill.

In designing this study, we made three assumptions. First, students will learn to use schema-broadening instruction as their approach to solving word problems: This has been shown in previous work (e.g., Fuchs et al., 2009). Second, we assumed that students with MD benefit from explicit instruction more than less structured instructional approaches: This has also been demonstrated in prior work, whereby students apply explicitly taught solution strategies better than they induce informal solution strategies (e.g., Kroesbergen et al., 2004). Third, we assumed that students with MD struggle with proper understanding of the equal sign. Although prior work has not investigated this issue, a substantial research base indicates that students with MD struggle across mathematics domains, from recalling basic facts to understanding place value to solving computation problems compared to non-MD peers (Hanich, Jordan, Dick, & Kaplan, 2001). On this basis, we assumed that equal-sign instruction would benefit students with MD.

Method

Participants

Participants were sampled from 51 third-grade classrooms in 18 schools. We screened 887 students with parental consent. Students who met the MD screening criterion on the Arithmetic subtest of the Wide Range Achievement Test-Revised (WRAT; Wilkinson, 1993; <26^th percentile) and the Math Problem Solving and Data Interpretation subtest of the Iowa Test of Basic Skills, Level 9, Form A (ITBS; Hoover, Dunbar, & Frisbie, 2001; <36^th percentile) were assessed individually on WRAT-Reading and on the 2-subtest Weschler Abbreviated Scale of Intelligence (WASI; The Psychological Corporation, 1999). Students scoring between the 26^th and 39^th percentile on WRAT-Reading or earning a T score below 30 on both WASI subtests were excluded. MD subtype was categorized as MD-only (<26^th percentile on WRAT-Arithmetic; <36^th percentile on ITBS; >39^th percentile on WRAT-Reading) or as MD concomitant with reading difficulty (MDRD; <26^th percentile on WRAT-Arithmetic; <36^th percentile on ITBS; <26^th percentile on WRAT-Reading). Ninety students met inclusion criteria.

Sixty of the 90 students were recruited as part of a larger study examining the efficacy of word-problem tutoring for students with MD.¹ The 60 students from this larger study were randomly assigned, blocking by MD subtype, to one of two conditions. The first was word-problem tutoring (n = 30). The other condition was control (no tutoring with conventional classroom instruction; n = 30). The proportion of MDRD (n = 21) and MD-only (n = 9) students in the word-problem tutoring and control groups was the same. After the first 60 students had been recruited, an additional 30 students were recruited using the same inclusion criteria, from other schools with similar demographics to the schools of the first 60 students. The proportion of MDRD and MD-only students recruited as part of the final group of 30 students was identical to the proportion in the word-problem tutoring and control groups. All of the additional 30 students were assigned to a third condition: word-problem plus equal-sign (combined) tutoring. During the course of the study, six combined tutoring students, three word-problem tutoring students, and one control student moved to different schools, at which time students were excluded from the study. Complete data were therefore available for 24 combined tutoring students, 27 word-problem tutoring students, and 29 control students.

The 60 students from the larger study were recruited from a pool of 667 students with parental consent; the additional 30 combined tutoring students were recruited from a group of 220 students with parental consent. The 30 students assigned to combined tutoring were recruited concurrently with the 60 students from the larger study. As discussed, our criterion for including students as MD was scoring <26^th percentile on WRAT-Arithmetic and <36^th percentile on ITBS. These criteria are similar, if not more stringent, than MD the cut-off points at the 25^th, 31^st, 35^th, or 45^th percentiles commonly used in other MD research studies (see Mazzocco, 2005). It is also important to note that based on the percentage of students who met study criteria among our students with parental consent, MD participants represent the lowest 13% of screened students even in a district with a high proportion of students receiving subsidized lunch. This phenomenon, whereby a cut-point on a commercial, standardized test yields a substantially smaller percentage of screened students than expected given the norms of the test, has been documented frequently in literatures on reading disability and MD (e.g., Fuchs et al., 2005). It indicates that the samples in these studies are similar to school-identified MD (and suggests an inadequate floor in the primary grades on many commercial, standardized achievement tests).

Students did not differ on demographics (sex, race, subsidized lunch status, special education status, English-language learner status, and retained status) as a function of condition. MD subtype was not associated with sex, race, subsidized lunch status, or English-language learner status, but was associated with special education status, χ²(1, N=90), 5.51, p = 0.019, and whether a student had been retained, χ²(1, N=90), 5.15, p = 0.023. These differences are expected given the greater level of severity associated with the MDRD subtype (Jordan, Hanich, & Kaplan, 2003). Including special education status or retention status in models did not alter the pattern of results. Also to describe students, teachers provided students' reading and math performance levels and completed the SWAN, a measure of students' inattention and hyperactivity/impulsivity (Swanson et al., n.d.). Table 2 presents demographic information, screening data, and teacher rating scale data by tutoring condition and MD subtype.

Table 2. Student Demographics, Screening Data, and Teacher Rating Data by MD Subtype and Tutoring Condition.

Variable	MDRD (n = 57)						MD-only (n = 23)						Across MD subtypes (n = 80)
	Word-problem tutoring (n = 18)		Combined tutoring (n = 18)		Control (n = 21)		Word-problem tutoring (n = 9)		Combined tutoring (n = 6)		Control (n = 8)		Word-problem tutoring (n = 27)		Combined tutoring (n = 24)		Control (n = 29)
	n	(%)	n	(%)	n	(%)	n	(%)	n	(%)	n	(%)	n	(%)	n	(%)	n	(%)
Males	12	(66.7)	9	(50.0)	13	(61.9)	6	(66.7)	2	(33.3)	3	(37.5)	18	(66.7)	11	(45.8)	16	(55.2)
Race
African American	12	(66.7)	12	(66.7)	11	(52.4)	4	(44.4)	4	(66.7)	4	(50.0)	16	(59.3)	16	(66.7)	15	(51.7)
White	1	(5.6)	3	(16.7)	1	(4.8)	1	(11.1)	1	(16.7)	1	(12.5)	2	(7.4)	4	(16.7)	2	(7.0)
Hispanic	4	(22.2)	0	(0.0)	6	(31.0)	4	(44.4)	0	(0.0)	3	(37.5)	8	(30.0)	0	(0.0)	9	(31.0)
Other	1	(5.6)	3	(16.7)	3	(28.6)	0	(0.0)	1	(16.7)	0	(0.0)	1	(3.7)	4	(16.7)	3	(10.3)
Reduced/free lunch	15	(83.3)	15	(83.3)	16	(76.2)	5	(55.6)	3	(50.0)	7	(87.5)	20	(74.1)	18	(75.0)	23	(79.3)
School-identified disability	8	(44.4)	4	(22.2)	4	(19.0)	1	(11.1)	0	(0.0)	0	(0.0)	9	(33.)	4	(16.7)	4	(13.8)
English-language learners	4	(22.2)	1	(5.6)	6	(28.6)	1	(11.1)	0	(0.0)	1	(12.5)	5	(18.5)	1	(4.2)	7	(24.1)
Retained	2	(11.1)	2	(11.1)	7	(33.3)	0	(0.0)	0	(0.0)	0	(0.0)	2	(7.4)	2	(8.3)	7	(24.1)
	M	(SD)	M	(SD)	M	(SD)	M	(SD)	M	(SD)	M	(SD)	M	(SD)	M	(SD)	M	(SD)
Screening Measures
WRAT-Arithmetic	20.94	(1.73)	20.89	(1.71)	21.14	(2.43)	20.89	(2.31)	21.17	(1.72)	20.25	(1.98)	20.93	(1.90)	20.96	(1.68)	20.90	(2.32)
WRAT-Reading	23.17	(2.73)	24.39	(2.81)	23.48	(3.37)	31.11	(2.15)	30.00	(1.10)	31.00	(2.14)	25.81	(4.57)	25.79	(3.50)	25.55	(4.58)
WASI IQ	83.83	(8.38)	81.94	(7.32)	82.81	(7.95)	92.44	(10.70)	91.00	(17.67)	93.13	(14.07)	86.7	(9.91)	84.21	(11.16)	85.66	(10.80)
ITBS	5.39	(2.12)	5.78	(1.52)	5.57	(2.25)	5.56	(1.01)	6.17	(1.17	6.50	(1.07)	5.44	(1.80)	5.86	(1.42)	5.83	(2.02)
Teacher Rating Scales
Reading level a	2.72	(.57)	2.83	(.38)	3.00	(.00)	2.11	(.93)	1.67	(.82)	1.88	(.83)	2.52	(.75)	2.54	(.72)	2.70	(.66)
Math level a	2.72	(.46)	2.89	(.32)	2.76	(.44)	2.33	(.50)	2.5	(.55)	2.50	(.53)	2.59	(.50)	2.79	(.41)	2.69	(.47)
SWAN b	3.67	(.89)	3.14	(.86)	3.83	(.97)	3.83	(1.21)	3.76	(1.84)	3.96	(.80)	3.72	(.99)	3.29	(1.17)	3.86	(.91)

^aAs determined by teacher: 1 = above grade level, 2 = at grade level, 3 = below grade level.

^bAs determined by teacher: 1 = far below, 2 = below, 3 = slightly below, 4 = average, 5 = slightly above, 6 = above, 7 = far above.

In addition to the 90 students with MD, 90 third-grade students without MD were administered the outcome measures to create a normative framework for understanding the performance of the students with MD. These 90 students, whom we sampled from six third-grade classrooms across five schools in which screening had occurred, comprised average and above average students and did not include any student who qualified for the tutoring study. We refer to this as the representative sample, for which WRAT-Arithmetic performance averaged 23.19 (SD = 2.67) and ITBS performance averaged 9.89 (SD = 4.20).

Screening Measures and Teacher Rating Scales

The WRAT-Arithmetic and the ITBS were administered in one 45-min whole-class screening session. On WRAT-Arithmetic, students have 10 min to answer 40 written computation problems of increasing difficulty. The tester reads directions aloud and then allows students to work on their own. Students answering four or fewer written problems correctly are administered 15 oral arithmetic problems individually. Students answering more than four written problems correctly are given 15 points without administration of the oral arithmetic problems. The maximum score is 55. As reported by Wilkinson (1993), median reliability for students aged 5-12 years is 0.94. On ITBS, students have 30 min to answer 22 written word problems with multiple-choice answers. Nine problems are standard word problems, and 13 problems require students to interpret and use information from graphs or pictures to answer word problems. The tester reads directions aloud and works three example problems to demonstrate responding to the multiple-choice format. Students then work individually. The maximum score is 22. At grades 1-5, Kuder-Richardson 20 is 0.83 to 0.87.

WRAT-Reading and the Vocabulary and Matrix Reasoning subtests of the WASI were administered in one 45-min individual session. On WRAT-Reading, students are provided with 42 words of increasing difficulty. Students read words aloud until reaching a ceiling of 10 consecutive errors. Students reading <5 words correctly name 15 letters. Students reading >4 words correctly are awarded 15 points without naming letters. The maximum score is 57. As reported by Wilkinson, median reliability for students aged 5-12 years is 0.94. On the WASI Vocabulary subtest, students name four pictures and define 37 words. Students define words until the end of the test or until reaching a ceiling of five consecutive errors. The maximum score is 78. On WASI Matrix Reasoning, students choose the best of five choices to complete a visual pattern. Students continue testing until completing all 35 items or until reaching a ceiling of 4 consecutive errors or 4 errors over 5 consecutive items. Vocabulary and Matrix Reasoning scores are combined to yield an Estimated Full Scale IQ. According to the test manual, median reliability for students aged 6-16 exceeds 0.92.

To describe the sample, classroom teachers estimated reading and math levels of each student (1 = above grade level; 2 = at grade level; 3 = below grade level) and completed the SWAN Rating Scale, an 18-item teacher rating scale (Swanson et al., n.d.). The SWAN comprises items reflecting the Diagnostic and Statistical Manual of Mental Disorders-IV (American Psychiatric Association, 1994) criteria for Attention-Deficit Hyperactivity Disorder: inattention and hyperactivity-impulsivity. Each item is rated on a 7-point scale. The SWAN correlates well with other dimensional assessments of behavior related to inattention (Swanson et al.). Coefficient alpha in this study was .97.

Outcome Measures

Three measures were administered at pre- and posttest. See Table 3 for sample items. Equal Sign Tasks (Matthews & Rittle-Johnson, 2009) assessed understanding of the equal sign and equality in a written format. First, the tester asks the student to define the equal sign. Then, the student decides whether each of eight nonstandard closed equations is correct. Next, students read four statements about equality and decide whether the statements are always true, sometimes true, or never true. Finally, students look at a closed equation with addends on both sides; break the equation into two parts; and define the meaning of the equal sign in the equation. On definitions of the equal sign, students are awarded 2 points for relational definitions (e.g., the same as), 1 point for equal definitions (e.g., something is equal), and 0 points for operational definitions (e.g., that number is the answer). On the other items, students receive 1 point for correct answers. The maximum score is 16. Coefficient alpha for the study sample was .73.

Table 3. Sample Items from Outcome Measures (Administered at Pre- and Posttest).

Equal Sign Tasks
What does the equal sign (=) mean?
3 + 2 = 6 - 1, correct or incorrect?
5 = 3 + 4, correct or incorrect?
“The equal sign means two amounts are the same.” Always, sometimes, or never true?
4 + 3 = 5 + 2. This problem has an equal sign in it. What does it mean here?
Open Equations
Standard equations	Nonstandard equations
__ + 3 = 7	__ = 4
2 + __ = 8	6 = 2 + __
__ - 4 = 3	__ = 6 + 5
9 - __ = 6	7 + 4 = __+ 6
8 - 3 = __	7 = __ - 2
	__ = 7 - 4
	12 - 5 = __ - 3
Story Problems
Total word problem		Generated equation
Andrea ran 4 miles last week and 5 miles this week. She rode her bike for 2 miles. How many miles did she run in all?		4 + 5 = X
Molly and Katie scored 10 goals in the soccer game. There were 8 girls on the team. Katie scored 6 goals. How many goals did Molly score?		X + 6 = 10
Benny made 14 baskets at the basketball game. The basket is 9 feet high. He made 6 baskets in the first half of the game. How many baskets did he make in the second half of the game?		6 + X = 14

With Open Equations (Powell, 2007), students have 10 min to solve 29 open equations. Twenty equations ask students to find the missing number in a standard (e.g., 3 + __ = 7) or nonstandard (e.g., 4 = 9 - __) format with three numbers. Seven equations involve finding the missing number in nonstandard addition and subtraction problems with four numbers (e.g., 3 + 6 = __ + 7 or 10 - __ = 8 - 3). Two equations ask students to find the missing number in a simple equality statement (4 = __). The score (maximum = 29) is the number of open equations answered correctly. Coefficient alpha for the study sample was .86.

Story Problems (Powell, 2007) comprises six brief word problems presented within a story format. Testers read each word problem and give approximately 1 min for students to respond. All six problems are Total problems (see below) with three numbers (a + b = c). Two problems require finding the total (c), two problems require finding the first part (a), and two problems finding the second part (b). The score (maximum = 6) is the number of correct math answers. Coefficient alpha for the study sample was .54.

Two research assistants independently entered responses on 100% of the test protocols for each outcome measure on an item-by-item basis into an electronic database, resulting in two separate databases. The discrepancies between the two databases were compared and rectified to reflect the student's original response. After discrepancies were rectified, student responses were converted into correct (1) and incorrect (0) scores using spreadsheet commands. This ensured 100% accurate scoring.

Core Classroom Mathematics Program

All classroom teachers used Houghton Mifflin Math (Greenes et al., 2005) to guide classroom mathematics instruction. Classroom word-problem instruction provided practice in applying problem-solution rules and emphasized computational requirements. Classroom instruction was explicit and relied on worked examples, guided group practice, independent work with checking, and homework. Houghton Mifflin Math does not provide an explicit definition of equal-sign instruction or instruction on the relational meaning of the equal sign. Equivalence is discussed in an addition regrouping lesson but not in lessons with equations.

Tutoring

Tutoring began the second week of October and ran for five weeks. Sessions were conducted 3 times per week (i.e., for 15 sessions) for 25 to 30 min a session. Tutors were 15 graduate students in education-related fields, one undergraduate student, and one retired elementary school teacher. Tutors participated in a 3-hour training to become familiar with and practice the combined tutoring and word-problem tutoring programs. Before each tutor's first tutoring session, tutors completed pseudo tutoring sessions with the coordinator of the project. Tutors met with the project coordinator at the end of the first and third weeks of tutoring to discuss tutoring and resolve student behavior issues.

Word-problem tutoring (schema-broadening instruction)

Word-problem tutoring occurred using a validated standard protocol called Pirate Math (Fuchs et al., 2009). In the present study, only the first two Pirate Math units were used: an introductory unit (five sessions) and the word-problem unit (10 sessions). The word-problem unit focused on the Total problem type in which two amounts are combined (e.g., Fred ate 3 pieces of cheese pizza and 2 pieces of mushroom pizza. How many pieces of pizza did Fred eat?). Although only one problem type (Total) was addressed in word-problem tutoring, we conceptualized tutoring as schema-broadening instruction because students were taught to broaden their schemas for Total problems to include problems with unexpectedly novel features such as irrelevant information or information presented in charts, graphs, or pictures.

During each word-problem tutoring session, five activities occur. The first activity is number combination flash cards. The tutor shuffles a deck of 200 addition and subtraction flash cards and shows cards one at a time. If the student's answer is correct, the tutor places the card in a correct pile. If the answer is incorrect, the tutor requires the student to use a counting up strategy (taught in the introductory unit) to find the correct answer. After the student counts up and answers correctly, the card is added to the correct pile. Flash cards are shown to the student for 1 min; at the end, the tutor counts the number of flash cards in the correct pile. Then, the student has another minute to try to beat the initial score. At the end of the second 1 min, the tutor counts the number of flash cards answered correctly.

The second activity is word-problem review. The student looks at a word problem from the previous session's final activity. The student talks aloud for approximately 30 sec about how to solve the problem. Because instruction on word problems does not begin until the sixth session, word-problem review begins with session seven.

The third activity is the daily lesson, which the tutor delivers guided by a script. The lesson is scripted to ensure tutors cover material in a similar manner. Tutors are encouraged to be familiar with scripts but are not permitted to read them verbatim. These daily lessons are explicit, and lessons 6 through 15 (i.e., in the Total word-problem unit) rely on schema-broadening instruction. In the introductory unit, the tutor provides instruction and practice on counting up addition and subtraction number combinations during session 1. Students are taught the min strategy for adding (start with the larger addend; count up the other addend; the answer is the last number counted) and the missing addend strategy for subtracting (start with the number after the minus sign; count up to the starting number; the answer is the number of counts or fingers up). Also in the introductory unit, the daily lesson for sessions 2-3 has the student work on double-digit addition and subtraction problems, respectively. Checking written work in terms of correct math, operation, and label is the focus of the daily lesson for the introductory unit's session 4. During session 5, the final daily lesson of the introductory unit, the tutor teaches the student how to solve addition algebraic equations with missing information in the third (e.g., 2 + 4 = X), second (e.g., 2 + X = 6), and first (e.g., X + 4 = 6) positions.

Sessions 6-15 focus on the Total word-problem type. In session 6, the tutor teaches a general problem-solving strategy: to read a word problem, to identify the question and important information, to ignore irrelevant information, to name the problem type, and to set up an algebraic equation that represents the word problem. For Total problems, students learn to identify two parts (P1 and P2) that combine for a total (T) within the word-problem narrative. So, the algebraic equation for the Total problem type is P1 + P2 = T. For example, students read the problem and underline the question, as in “Katie went to the pet store and counted 9 dogs and cats. If 4 of the animals were dogs, how many cats did Katie count?” Students then decide the problem type. This problem is a Total problem because two amounts (dogs and cats) are combined to make a Total. Students next write an algebraic equation (4 + X = 9) to represent the problem and solve the equation (X = 5) to solve the word problem. During each daily lesson in sessions 6-15, the tutor and student work through three Total word problems. Total problems with X in the third position are introduced in session 6. Second- and first-position Total problems are added to the student's word-problem repertoire in sessions 9 and 10, respectively. Word problems in sessions 13, 14, and 15 require the student to find important information to solve a word problem from scenes, charts, or graphs. The tutor provides explicit and scaffolded instruction in every daily lesson.

The fourth activity is word-problem flash cards. The tutor shows and reads word-problem flash cards; the student names the problem type and places each card on a sorting mat to distinguish Total problems from unknown problem types. Flash cards are presented for 2 min. Then, the tutor counts the number of correctly sorted cards and reviews up to 5 errors. Similar to the word-problem review, the student does not start working with word-problem flash cards until word problems are introduced during session six.

The fifth activity is final review, where the student works independently for up to 4 min to solve nine standard addition algebraic equations (e.g., X + 4 = 9, 3 + X = 10, 2 + 7 = X) and one word problem. The tutor grades the review for accuracy and keeps the review for use in the next session's word-problem review.

Word-problem plus equal-sign (combined) tutoring

In addition to these Pirate Math lessons, combined tutoring students received relational equal-sign instruction in every session. This instruction occurred in each Pirate Math session after number combinations flash cards (word-problem tutoring activity 1) and before the word-problem review (word-problem tutoring activity 2). During the first five sessions, explicit instruction focused on the relational meaning of the equal sign (Rittle-Johnson & Alibali, 1999). In session 1, students were introduced to equality vocabulary: equal, same, and sides. Students were taught to interpret and read the equal sign as “the same as.” During sessions 1-5, manipulative bears and blocks along with representational pictures on paper were used to teach these equality concepts. In every session, the student worked six problems and determined “if this side (left side of equation) is the same as that side (right side of equation).”

During the Total word-problem type unit (i.e., the next 10 sessions in the same time slot between word-problem activity 1 and word-problem activity 2), students were presented with six closed equations, and the tutor asked the student to decide whether the equation was appropriate. On these closed equations for each lesson, the student was presented with one identity statement (e.g., 5 = 5), one equality statement (e.g., 3 + 4 = 5 + 2), two standard equations (e.g., 7 − 2 = 5), and two nonstandard equations (e.g., 3 = 7 − 4). It is important to note that students worked only on closed equations during equal-sign instruction. No instruction on open equations or how to solve open equations was provided. Students crossed out inappropriate equations (e.g., 4 = 9 + 5). Every day, for all equations, the tutor reviewed a work-checking strategy, which involved the student asking, “Is this side the same as this side?” Equal-sign instruction during each session lasted approximately 2 to 4 min. After equal-sign instruction concluded in each session, combined tutoring students participated in a daily lesson. The daily lesson was identical in both conditions except in one way: During the Total problem type lessons (sessions 6-15), tutors reminded students to check “if this side was the same as that side” for all algebraic equations generated from word problems. Following each session's daily lesson, combined tutoring students did the same word-problem flash cards.

To ensure word-problem and combined tutoring sessions were similar in duration (given the extra activity for combined tutoring), the final review activity of each session differed. Whereas word-problem tutoring students worked on nine algebraic equations (e.g., X + 2 = 7, 3 + X = 9, 4 + 3 = X) and one word problem for 4 min, combined tutoring students worked on just one word problem for 2 min. The tutor scored the word problem and kept it for the following session's word-problem review.

It is important to note that word-problem tutoring students received more practice (through the final review activity) on solving equations than combined tutoring students. Whereas combined tutoring students were exposed to standard and nonstandard closed equations during tutoring, these students never practiced solving open equations except in the context of solving word problems. By contrast, word-problem tutoring students practiced solving open equations both in the context of solving word problem equations and working on the final review activity.

Fidelity

To evaluate fidelity of treatment implementation, all sessions were audiotaped, and 17.26% of sessions were randomly sampled to ensure comparable representation of tutoring conditions, tutors, and lessons. Three research assistants listened independently to tapes while completing a checklist to identify essential points addressed during the session. Fidelity averaged 98.39% (SD = 3.36) for word-problem tutoring and 94.74% (SD = 4.45) for combined tutoring.

Procedure

Whole-class screening (e.g., WRAT-Arithmetic and ITBS) was conducted in one 30-min session the first week of September. Individual screening (e.g., WASI and WRAT-Reading) took place in one 45-min session during the last two weeks of September. Pretesting on the outcome measures occurred in one 30-min individual session during the first week of October, approximately 4 to 6 days before tutoring began. Posttesting on the outcome measures occurred in one 30-min individual session 4 to 6 days after the last session was conducted. All control students were pre- and posttested in the same time frame as tutored students. The representative sample was administered the WRAT-Arithmetic and ITBS in one 30-min session the first week of September. Equal Sign Tasks, Open Equations, and Story Problems were administered in one 30-min group session during the second or third week of October.

Data Analysis

To assess pretreatment comparability of treatment groups, we applied two-way ANOVAs to the screening and teacher rating scale data and to pretreatment scores using tutoring condition (combined tutoring vs. word-problem tutoring vs. control) and MD subtype (MDRD vs. MD-only) as the factors. To assess learning as a function of tutoring condition and MD subtype, we ran preliminary two-way ANOVAs to improvement scores, which revealed no significant interactions between MD subtype and tutoring condition. On this basis, we trimmed MD subtype from our model and then tested a set of orthogonal contrasts to assess our five hypotheses using Helmert coefficients (Hinton, Brownlow, McMurray, & Cozens, 2004) within a General Linear Model. Helmert contrasts first assessed whether one condition differed from the two remaining conditions; then assessed whether the remaining conditions differed from each other. We calculated ESs by subtracting means and dividing by the SD (Hedges & Olkin, 1985). Finally, to explore the tenability of our proposed causal mechanism by which effects accrued for the combined tutoring condition, we conducted mediation analysis (Baron & Kenny, 1986).

Results

See Table 2 for screening and teacher rating scale data. Table 4 displays pretest, posttest, and improvement scores on the outcomes measures by tutoring condition and MD subtype.

Table 4. Pre- and Posttest Performance on Outcome Measures.

	MDRD (n = 57)						MD-only (n = 23)						Across MD subtypes (n = 80)
	Word-problem tutoring (n = 18)		Combined tutoring (n = 18)		Control (n = 21)		Word-problem tutoring (n = 9)		Combined tutoring (n = 6)		Control (n = 8)		Word-problem tutoring (n = 27)		Combined tutoring (n = 24)		Control (n = 29)
	M	(SD)	M	(SD)	M	(SD)	M	(SD)	M	(SD)	M	(SD)	M	(SD)	M	(SD)	M	(SD)
Equal Sign Tasks
Pretest (overall score)	6.11	(1.94)	6.89	(2.19)	6.95	(2.54)	7.33	(3.08)	8.17	(2.56)	8.00	(3.51)	6.52	(2.39)	7.21	(2.30)	7.24	(2.81)
Posttest (overall score)	6.28	(1.84)	13.72	(1.60)	6.29	(1.82)	8.44	(3.09)	12.67	(3.44)	7.50	(3.16)	7.00	(2.50)	13.46	(2.17)	6.62	(2.27)
Improvement	0.17	(1.95)	6.83	(2.26)	-0.67	(2.37)	1.11	(3.30)	4.50	(2.43)	-0.50	(5.18)	0.48	(2.46)	6.25	(2.47)	-0.62	(3.28)
Open Equations
Pretest (standard equations)	2.50	(2.33)	3.67	(2.43)	3.76	(2.36)	4.89	(2.85)	5.83	(1.72)	2.63	(2.33)	3.30	(2.71)	4.21	(2.43)	3.45	(2.37)
Posttest (standard equations)	4.28	(2.40)	4.11	(2.49)	4.52	(2.32)	6.22	(2.64)	6.50	(1.87)	4.63	(3.62)	4.93	(2.60)	4.71	(2.54)	4.55	(2.68)
Improvement	1.78	(2.37)	0.44	(2.38)	0.76	(1.58)	1.33	(2.24)	0.67	(1.21)	2.00	(2.62)	1.63	(2.29)	0.50	(2.13)	1.10	(1.95)
Pretest (nonstandard equations)	2.39	(2.78)	3.00	(2.93)	3.62	(2.84)	4.00	(2.60)	4.00	(2.76)	2.88	(1.96)	2.93	(2.46)	3.25	(2.86)	3.41	(2.61)
Posttest (nonstandard equations)	4.33	(3.03)	6.17	(3.45)	4.19	(3.27)	5.67	(2.55)	10.83	(4.88)	3.25	(2.92)	4.78	(2.90)	7.33	(4.27)	3.93	(3.15)
Improvement	1.94	(2.67)	3.17	(3.67)	0.57	(2.60)	1.67	(2.00)	6.83	(4.45)	0.38	(2.92)	1.85	(2.43)	4.08	(4.11)	0.52	(2.64)
Story Problems
Pretest (missing information before equal sign)	0.39	(0.61)	0.22	(0.43)	0.48	(0.98)	0.44	(1.33)	0.50	(0.84)	0.50	(0.53)	0.41	(0.89)	0.29	(0.55)	0.48	(0.87)
Posttest (missing information before equal sign)	0.61	(0.78)	0.72	(1.23)	0.52	(0.87)	1.11	(1.27)	1.50	(1.05)	0.13	(0.35)	0.78	(0.97)	0.92	(1.21)	0.41	(0.78)
Improvement	0.22	(0.88)	0.50	(1.25)	0.05	(0.97)	0.67	(1.22)	1.00	(1.55)	-0.38	(0.52)	0.37	(1.01)	0.63	(1.31)	-0.07	(0.88)
Pretest (missing information after equal sign)	0.56	(0.62)	0.72	(0.57)	0.48	(0.51)	0.56	(0.53)	0.67	(0.52)	0.75	(0.71)	0.56	(0.58)	0.71	(0.55)	0.55	(0.57)
Posttest (missing information after equal sign)	1.00	(0.84)	1.06	(0.73)	0.95	(0.74)	1.33	(0.87)	1.17	(0.75)	0.75	(0.71)	1.11	(0.85)	1.08	(0.72)	0.90	(0.72)
Improvement	0.44	(1.04)	0.33	(0.97)	0.48	(0.75)	0.78	(1.09)	0.50	(0.84)	0.00	(0.93)	0.56	(1.05)	0.38	(0.92)	0.34	(0.81)

Pretest Comparability on Screening Measures, Teacher Rating Scales, and Outcome Measures

At pretest, there were no significant differences among tutoring conditions on any screening measure: WRAT-Arithmetic, ITBS, WRAT-Reading, or WASI. As expected, however, there were significant differences between MDRD and MD-only students on WRAT-Reading (p < 0.001) and WASI (p < 0.001). Because students were placed into MDRD and MD-only subtypes based on the WRAT-Reading, the significant difference demonstrates that these MD subtypes are indeed two separate groups, as intended. The significant difference between MDRD and MD-only students on the WASI was expected because the WASI relies heavily on vocabulary skills, which is related to reading skill. WASI differences between MD subtypes have been found in other samples (e.g., Powell, Fuchs, Fuchs, Cirino, & Fletcher, 2009). None of the interactions between tutoring condition and MD subtype was, however, significant for any of the four screening measure; so, the main effects for MD subtype on WRAT-Reading and on WASI do not threaten the validity of the study.

There were no significant pretest differences on the SWAN. On teacher ratings of student reading level, there were significant differences between MD subtypes corroborating the WRAT-Reading test differences of MDRD and MD-only students. Neither the tutoring condition main effect nor the interaction was significant. Interestingly, teacher ratings of student math level also showed significant differences between MD subtypes.² More importantly, however, none of the differences among tutoring conditions and none of the interactions between tutoring conditions and MD subtype was significant. There were also no significant differences on pretest performance on Equal Sign Tasks, Open Equations, or Story Problems.

Improvement on Outcome Measures

On the improvement score for each outcome measure, there were no significant differences between MDRD and MD-only students and no significant interactions between tutoring condition and MD subtype. On this basis, we trimmed MD subtype from the model and conducted orthogonal contrasts aligned with our hypotheses.

Equal Sign Tasks

On Equal Sign Tasks, improvement as a function of tutoring condition was significant F(2, 77) = 44.50, p < 0.001. As hypothesized (COMB > WP = CON), Helmert contrasts indicated that (a) combined tutoring students improved significantly more than word-problem tutoring and control students (p < 0.001: ESs = 2.35 and 2.34, respectively), and (b) word-problem and control students improved comparably (p = 0.143; ES = 0.38).

Open Equations

On standard open equations, the difference among tutoring conditions was not significant, F(2, 77) = 1.79, p = 0.172, providing the basis to reject this hypothesis (COMB = WP > CON). By contrast, in line with our hypothesis concerning nonstandard open equations (COMB > WP = CON), the difference among tutoring conditions was significant, F(2, 77) = 8.803, p < 0.001. Helmert contrasts indicated that combined tutoring students outperformed word-problem tutoring and control students (p < 0.001; ESs = 0.67 and 1.06, respectively), but there was no significant difference between word-problem tutoring and control students (p = 0.111; ES = 0.53).

Story Problems

On word problems with missing information after the equal sign, improvement as a function of tutoring condition was not significant, F(2, 77) = 0.407, p = 0.667, providing the basis to reject this hypothesis (COMB = WP > CON). On the other hand, for problems with missing information before the equal sign, the difference among tutoring conditions approached statistical significance, F(2, 77) = 2.90, p = 0.061, with Helmert contrasts lending support for our hypothesis (COMB > WP = CON): Combined tutoring marginally outperformed word-problem tutoring and control conditions (p = 0.073; ESs = 0.22 and 0.63, respectively), but the difference between word-problem tutoring and control students was not significant (p = 0.128; ES = 0.47).

Assessing the Proposed Causal Mechanism by Which Combined Tutoring Effects Accrued

To test our proposed causal mechanism (i.e., equal-sign instruction exerts a positive influence on solving equations, which in turn enhances word-problem solving, we conducted mediation analyses in four steps (Baron & Kenny, 1986) using regression analysis. Because we were interested specifically on the effect of equal-sign tutoring on solving equations and word problems, the most salient outcome and mediator variables on which to focus were, respectively, performance on word problems with missing information before the equal sign (referred to below as word problems) and performance on nonstandard equations (referred to below as nonstandard equations).

In the first step of the mediation analysis, we assessed the effects of tutoring condition on the outcome (posttest word-problem performance), controlling for pretest word-problem performance. Model 1 in Table 5 shows that tutoring condition accounted for unique variance in explaining posttest word-problem performance, even when controlling for pretest word-problem performance. In the second step, we assessed the effects of tutoring condition on the mediator (posttest nonstandard-equation performance), controlling for pretest nonstandard-equation performance. Model 2 in Table 5 shows that tutoring condition accounted for unique variance in explaining the mediator even when controlling for pretest nonstandard-equation performance. These findings essentially repeat the effects for tutoring condition in the earlier analyses.

Table 5. Mediation Analysis for the Proposed Causal Mechanism.

Model	R2	SEE	R2	F	p-value	B	SE	Beta	t	p -value
1: Tutoring on word-problems posttest	0.36	0.95	0.13	5.75	0.005
Constant						1.14	0.29		3.92	<0.001
Word problems (pretest)						0.38	0.14	0.30	2.77	0.007
Tutoring						-0.29	0.13	-0.24	-2.22	0.029
2: Tutoring on nonstandard-equations posttest	0.61	2.98	0.37	22.19	<0.001
Constant						6.66	0.99		6.75	<0.001
Equation solving (pretest)						0.68	0.13	0.48	5.27	<0.001
Tutoring						-1.74	0.41	-0.38	-4.23	<0.001
3: Nonstandard equations on word-problems posttest	0.52	0.87	0.28	9.60	<0.001
Constant						-0.11	0.18		-0.61	0.546
Word problems (pretest)						0.14	0.14	0.11	1.00	0.318
Equation solving (pretest)						0.09	0.05	0.24	2.01	0.048
Equation solving (posttest)						0.09	0.03	0.32	2.90	0.005
4: Tutoring and nonstandard equations on word-problems posttest	0.54	0.87	0.29	7.63	<0.001
Constant						0.28	0.37		0.77	0.443
Word problems (pretest)						0.16	0.14	0.13	1.17	0.246
Equation solving (pretest)						0.10	0.05	0.26	2.22	0.029
Equation solving (posttest)						0.07	0.03	0.25	2.06	0.043
Tutoring						-0.17	0.13	-0.13	-1.23	0.223

In the third step, however, we assessed the effects of the mediator on outcome, controlling for pretest word-problem performance and controlling for pretest nonstandard-equation performance. Model 3 in Table 5 shows that the mediator variable did in fact account for unique variance in explaining the outcome when controlling for pretest word-problem performance and pretest nonstandard-equation performance. In the final and critical step of the mediation analysis, we assessed the effects of tutoring condition and the mediator (posttest nonstandard-equation performance) on the outcome (posttest word-problem performance), controlling for pretest word-problem performance and pretest nonstandard-equation performance. Model 4 in Table 5 shows that once the mediator was accounted for as a predictor in the model, tutoring condition was no longer significant, even as the mediator retained its significance. This set of analyses demonstrates that performance on nonstandard equations mediates the effect of tutoring condition in explaining posttest word-problem performance.

Comparability to the Representative Sample

As expected, significant differences favored the pretest performance of the representative sample over the MD students, regardless of tutoring condition, on Equal Sign Tasks, Open Equations, and Story Problems. By contrast, at posttest, there were some significant effects. Follow-up tests that adjusted the p-value to 0.0083 to account for six contrasts per measure indicated that on Equal Sign Tasks, combined tutoring students demonstrated superior performance over the representative sample (p < 0.001; ES = 1.92) even as the representative sample still significantly outperformed MD word-problem tutoring students (p = 0.005; ES = 0.60) and control students (p < 0.001; ES = 0.76). On Open Equations, the performance of combined tutoring and representative-sample students was comparable (p = 0.344), although the representative sample significantly outperformed word-problem tutoring students (p = 0.005; ES = 0.63) and control students (p < 0.001; ES = 0.83). On Story Problems, however, the representative sample significantly outperformed MD students, regardless of treatment condition.

Discussion

The purpose of this study was to assess the efficacy of explicit equal-sign instruction for third-grade students with MD. Direct effects of this instruction were measured using the Equal Sign Tasks, whereas transfer effects were examined on Open Equations and Story Problems. Equal-sign instruction was contextualized within a schema-broadening word-problem tutoring program (i.e., Pirate Math) and compared against two competing conditions: word-problem tutoring alone (without equal-sign instruction) and no-tutoring control. Inclusion of the word-problem tutoring condition allowed us to assess whether instruction on equal-sign understanding transferred to enhance performance on open equations and word problems or whether, in fact, improvement on these measures was more simply due to word-problem instruction. The no-tutoring control condition was included to control for history and maturation effects.

Equal Sign Tasks assessed understanding of the equal sign through definitions and closed equations. Combined tutoring provided explicit instruction on a relational definition of the equal sign as well as experience in determining whether closed equations are acceptable. As hypothesized, because of this explicit instruction, combined tutoring students (who received equal-sign instruction) demonstrated superior learning over word-problem tutoring and control students, with ESs of 2.35 and 2.34, respectively. Impressively, combined tutoring students also demonstrated superior performance over the representative sample, with a posttest ES of 1.92. At pretest, less than 10% of all students provided a relational definition of the equal sign. At posttest, almost all combined tutoring students provided definitions of the equal sign such as same as and this side is the same as that side. By contrast, students without equal-sign instruction provided definitions such as what the sum is, the number is the answer, take away, to put the total, and an answer. These conceptually flawed definitions are similar to the definitions and understandings documented by Kieran (1981) and McNeil and Alibali (2005). In keeping with McNeil and Alibali, our explicit equal-sign instruction changed student understanding of the equal sign from the flawed definition as an operational symbol to the correct definition as a relational symbol. Our first hypothesis (COMB > WP = CON) was corroborated.

We were also interested in whether improved understanding of the equal sign as a function of equal-sign instruction transferred to performance on open equations (which was not taught as part of equal-sign instruction). If students understand the equal sign as a symbol that represents a relationship between the numbers on each side of the equal sign (Jacobs et al., 2007), they should try to make each side of an equation the same instead of merely finding a total. Previous experimental work demonstrated equal-sign instruction had a positive impact on solving open equations for average-performing students (McNeil & Alibali, 2005; Rittle-Johnson & Alibali, 1999). Our study sought to extend this line of work to another population, students with MD.

On our Open Equations task, students solved equations with missing information. No explicit instruction on solving open equations was provided in combined tutoring. Yet, students in both active tutoring conditions, combined tutoring and word-problem tutoring alone, did receive instruction on solving algebraic number sentences as part of the word-problem tutoring protocol, where students learned to represent the underlying structure of a word problem using an algebraic equation and then solve for X. We note, however, that this word-problem tutoring only addressed writing and solving standard equations (i.e., 4 + X = 9 or 7 − 2 = X), not nonstandard equations (i.e., 8 + 3 = X + 2 or 5 = 8 − X). Also, the Open Equations measure was formatted in such a way that students solved open equations with a blank not a variable. On standard open equations, in contrast to our hypothesis that both active conditions would outperform control students (COMB = WP > CON), there were no significant differences among tutoring conditions.

Nonstandard equations were, however, of greater interest because neither tutoring condition provided instruction or practice on this type of equation. Moreover, because school textbooks and instruction rely heavily on instruction using standard equations (Capraro et al., 2007; McNeil et al., 2006), we assumed students had little to no exposure on nonstandard equations. On the nonstandard equations, the benefits of equal-sign instruction were pronounced, confirming our third hypothesis (COMB > WP = CON). ESs for combined tutoring students over word-problem tutoring and control students were 0.67 and 1.06, respectively and, importantly, the difference between word-problem tutoring and control students was not statistically significant. In these ways, results on nonstandard equations suggest that a relational understanding of the equal sign carries important transfer effects to solving nonstandard equations. Because many mathematical skills, such as algebra and word problems, require students to think outside the box and solve nonstandard equations, the significant equal-sign tutoring effects reveal the benefit of understanding the equal sign in a relational manner.

To explore the possibility of transfer from equal-sign instruction to word-problem performance, we looked separately at the items on the Story Problems where equations involved missing information before versus after the equal sign. As previously demonstrated (e.g., Fuchs et al., 2009; Fuchs, Seethaler et al., 2008), equations with missing information after the equal sign are easier for students to solve because they are more routine (Behr, Erlwanger, & Nichols, 1980). Also, if missing information occurs after the equal sign, students can interpret the equal sign in an operational manner (i.e., find the total or do something) or a relational manner and still derive a correct answer (McNeil et al., 2006). Analysis of word problems with missing information before the equal sign should therefore provide greater insight into the effect of equal-sign instruction on word problems.

On word problems with missing information after the equal sign, there were no significant differences among tutoring conditions, providing the basis to reject our fourth hypothesis (COMB = WP > CON). This absence of effects on word problems with missing information after the equal sign prohibits us from inferring that transfer from equal-sign instruction to word-problem performance occurred. Prior schema-broadening work (Fuchs et al., 2009; Fuchs, Seethaler et al., 2008) using Pirate Math tutoring that did demonstrate significant gains for word-problem tutoring over control students delivered tutoring for 39-45 sessions and taught three word-problem types. By contrast, word-problem and combined tutoring students in the current study received only 15 sessions of Pirate Math and focused on only one word-problem type. We expected, based on the brief equal-sign instruction provided by McNeil and Alibali (2005) and Rittle-Johnson and Alibali (1999), students would benefit from equal-sign instruction provided over 15 days of tutoring. We provided the first 15 Pirate Math lessons to gain insight into the possible effect of equal-sign instruction on word-problem performance. It is possible that with the full Pirate Math tutoring protocol, results favoring word-problem tutoring over control students would have emerged. Even so, given the absence of effects on word problems with missing information after the equal sign, it is even more interesting that on word problems with missing information before the equal sign, combined tutoring students – and only students in this condition – demonstrated significant improvement over control students, with a large ES of 0.63.

Finally, to probe deeper into the possibility that equal-sign understanding extends to word problems, we proposed a causal mechanism. We suggested that if students received equal-sign instruction and became more effective at solving open equations, then they should better understand how algebraic equations represent word-problem schema and they should generate equations more accurately. Moreover, students should solve algebraic equations derived from word problems more efficiently which, in turn, should make students better at solving word problems. Research supports the efficacy of generating equations to represent word problems at promoting superior problem solving (Carpenter et al., 1988; Fuchs, Seethaler et al., 2008) but, to our knowledge, no research has examined the effect of equal-sign instruction on successful word-problem solving. The mediation analysis corroborated our causal mechanism. It showed that once the mediator (i.e., nonstandard-equation performance) was accounted for as a predictor in the model, tutoring condition was no longer significant, even as the mediator retained its significance. This set of analyses demonstrates that performance on nonstandard equations mediates the effect of tutoring condition in explaining posttest word-problem performance.

Before we conclude, we note our study's limitations. First and foremost, we did not randomly assign the 90 students to the three study conditions. The word-problem tutoring and control students were randomly assigned as part of a larger study. The 30 combined tutoring students were recruited after recruitment and random assignment of the larger study had been completed. The 30 students comprising this condition were, therefore, recruited in a separate process, from schools with similar demographics to the schools of the original 60 students. We analyzed screening and pretest measures along with demographic information for differences among conditions, and we did not find any unexpected differences. Students in the three conditions performed comparably on WRAT-Arithmetic, ITBS, Equal Sign Tasks, Open Equations, and Story Problems. Students in the three conditions represented the two MD subtypes in the same proportion; performed comparably on WASI and WRAT-Reading; and were comparable on all student demographics. Consequently, we have no reason to suspect that extraneous variables account for findings. Without random assignment, however, we cannot assume the three conditions were not different on some other, unmeasured dimension. Additional limitations concern our outcome measures. We asked students to demonstrate their understanding of the equal sign only in a written format. Moreover, we did not assess maintenance, or follow-up performance, without which we are unsure if our treatment effects were temporary or long lasting. Future research would be strengthened by incorporating random assignment; conducting oral interviews or think-aloud protocols asking students to provide step-by-step explanations when they solve open equation and word problems to gain insight into how they conceptualized and use the equal sign; and administering maintenance tests weeks or months after conclusion of tutoring.

In sum, our results on Equal Sign Tasks indicate that explicit equal-sign instruction positively impacts understanding of the equal sign for students with MD and transfers to performance on solving nonstandard open equations. Equal-sign instruction in this study took place over 15 sessions for 2-4 min each session. Future research should investigate whether similar results can be obtained over fewer sessions. Also, the content of the explicit equal-sign instruction should be examined to understand whether students need instruction through the use of manipulatives, pictorial representations, and work with abstract problems (Hudson & Miller, 2006) and whether instruction using closed or open equations and standard or nonstandard equations (or a combination of all these equation types) is more efficient for influencing equal-sign understanding as well as important forms of transfer. Furthermore, future studies should use larger samples with students across the elementary grades and compare effects of equal-sign instruction for students with and without MD. Other studies should compare the effects of equal-sign instruction provided within individual or small-group tutoring to equal-sign instruction provided at the classroom level.

Results on word problems with missing information before the equal sign suggest that a relational understanding of the equal sign transfers to word-problem solving. This transfer effect may not be as pronounced as the transfer effects to nonstandard open equations, but our results lay the groundwork for future work assessing transfer effects for concurrent equal-sign and word-problem instruction. In this vein, future research should explore whether explicit equal-sign instruction should be embedded within explicit word-problem instruction focused on setting up an algebraic equation and solving for a missing variable. (Our equal-sign instruction and word-problem instruction occurred in the same session, but an explicit connection between to two types of instruction was not made.) Also, the effects of equal-sign instruction embedded within word-problem instruction on other types of word problems besides the Total word-problem type addressed in the present study (e.g., Difference or Change) should be examined to assess if equal-sign instruction is beneficial across word-problem types. Finally, future research should consider whether understanding the equal sign in a relational manner is an integral part of being able to solve word problems. That is, are there other instructional components that are more important to a successful word-problem tutoring program above and beyond equal-sign instruction.