The Psychophysics of Algebra Expertise: Mathematics Perceptual Learning Interventions Produce Durable Encoding Changes

Abstract

Mathematics requires thinking but also pattern recognition. Recent research indicates that perceptual learning (PL) interventions facilitate discovery of structure and recognition of patterns in mathematical domains, as assessed by tests of mathematical competence. Here we sought direct evidence that a brief perceptual learning module (PLM) produces changes in basic information extraction. Accuracy and speed of undergraduate participants’ encoding of equations was assessed in a psychophysical task at pretest and delayed posttest. In between, the experimental group completed an Algebraic Transformations PLM, which involved identifying valid transformations of equations. Relative to controls, PLM participants showed reliable changes in encoding equations, detectable psychophysically 24 hours later. Encoding improvements were shown robustly by participants who were initially less proficient at algebra and were negligible for participants who were initially proficient. These results provide direct evidence for durable changes in information encoding produced by a PL intervention targeting a complex mathematical skill.

Introduction

How do students in an algebra class differ from their teacher in solving problems? What cognitive changes must occur for students to become proficient? Typical answers would be that the teacher knows and imparts to students facts, concepts and procedures. But the teacher also sees algebraic structures and representations differently. A primary driver of expertise in mathematics and many domains is perceptual learning (PL) – domain specific changes in the extraction of information (Gibson, 1969; Kellman & Massey, 2013). A well-known example is that chess masters more effectively encode structure in board positions than do novices (Chase & Simon, 1973). More generally, it has been argued (Kellman, 2002) that PL produces a variety of effects that fall into two categories: 1) discovery effects, such that information pickup becomes more selective, and perceivers discover new relationships, and 2) fluency effects, including faster encoding and reduced cognitive load (Kellman, 2002; Kellman & Garrigan, 2009). Recent work suggests that PL plays an important role in high-level cognitive domains, even symbolic ones such as mathematics (e.g., Kellman & Massey, 2013; Kellman, Massey, & Son, 2010; Landy & Goldstone, 2007). PL is not addressed systematically by conventional instructional methods. In mathematics, PL may develop over time and experience with the materials (rather than through direct instruction); however, students encounter many obstacles to effective PL due to infrequent opportunities to explicitly focus on the structural patterns that signal which concepts and procedures can be applied.

Previous Research

Recent PL research has revealed a great deal about the conditions under which such learning occurs (for a review, see Kellman & Garrigan, 2009). Kellman & colleagues have incorporated principles of PL into learning interventions aimed at accelerating PL in complex cognitive domains (Kellman, 2013; Kellman & Kaiser, 1994; Kellman et al., 2010). Perceptual learning modules (PLMs) involve short and varied classification trials with feedback; they can be enhanced by using adaptive learning techniques that arrange spacing and mastery criteria in learning based on both accuracy and reaction times (Mettler & Kellman, 2013; Mettler, Massey, & Kellman, 2011). In an Algebraic Transformations PLM (Kellman et al., 2010), eighth and ninth graders in Algebra I classes at mid-year mapped target equations to legal transformations among distractor illegal transformations but did not practice solving equations. Pretest and posttests tested both the mathematical transformation task used in the PLM and transfer to algebra equation solving. These students had acquired a good basic grasp of algebra concepts and procedures before the PLM, as evidenced by 80% average accuracy in solving simple equations such as X + 8 = 12 on the pretest, but they were poor at seeing structure and potential transformations of equations, taking around 28 seconds per problem. After two 35–40 minute sessions with the PLM, students solved equations markedly faster, reducing their solving time by more than 55% to about 12 s, a gain that was fully maintained at a two-week delay. Characteristics of both the intervention and the results implicated perceptual learning as the cause of the improvement in students’ performance (Kellman et al., 2010). Students were not explicitly taught any new rules or principles of algebra in the PLM, nor did they practice solving problems, yet practice at seeing transformations increased their problem-solving efficiency.

These results complement others showing the importance of seeing in mathematics learning (Kulp et al., 2004; Landy & Goldstone, 2007; Ottmar, Landy, & Goldstone, 2012). That the relevant encoding and pattern extraction skills can be accelerated by PLMs appears to be true across many learning domains (Kellman & Kaiser, 1994; Kellman et al., 2008; Krasne, Hillman, Kellman, & Drake, in press). These results have shown large effect sizes and gains that persist over substantial delays (Drake et al., in press; Kellman et al., 2010; Massey et al., 2010; Mettler & Kellman, 2013). Assessments of PLMs’ efficacy have typically focused on domain-relevant tasks, such as math problem solving.

Because PLMs aim at improving information extraction, it is interesting to ask whether learners who have used PLMs in complex tasks show measurable, lasting changes in basic encoding of information, as measured using psychophysical methods. After a PLM focusing on seeing transformations in algebra, for example, one might observe not only improved mathematics performance, but also improved speed or accuracy of encoding, comparison, or discrimination of mathematical objects.

One study (Thai, Mettler, & Kellman, 2011) examined basic information extraction consequences of a PL intervention on an immediate posttest. The authors trained Chinese-illiterate participants on two PLMs involving Chinese characters. On each trial, learners selected which of two characters shared either a feature (radical) or the overall structure (configuration) of a target character. Completing a PLM produced significant improvement in visual search relative to controls, and the particular kinds of improvement observed depended on which PLM (feature or structure) each learner had completed.

Current Study

Here we sought to find evidence for durable changes in information encoding and sensitivity to structure in a mathematical domain. We used the Algebraic Transformations PLM previously employed by Kellman et al. (2010), because it showed strong mathematics PL effects in classroom settings.

A difficulty in looking for basic encoding changes after use of a high-level, domain-specific PLM is that we do not initially know what kinds of changes to look for. The visual search task used by Thai et al. (2011) would be impractical for algebra. We do know that the Algebraic Transformations PLM (Kellman et al., 2010) focuses on students’ processing of relations and transformations. It could lead to more rapid or accurate encoding of equations or their parts, chunking, and/or improved comparison abilities. One could instead look for improved discrimination of numerals or characters such as x or y, but this seems less intuitively connected to equation structure or transformation. We note that the choice of a task to detect particular encoding changes was made intuitively from a large set of possibilities.

We developed a psychophysical task involving speeded judgment of two simultaneously presented equations as same or different. We predicted that participants who completed the PLM would show some evidence of improved abilities to rapidly compare equations when tested 24 hours later. For this study, we used an undergraduate population, all of whom had previous classroom experience with and have demonstrated competence in algebra (“Profile of Admitted Freshmen” – UCLA, 2013). It was possible that encoding abilities relevant to algebraic transformations would be quite advanced in this subject group, such that a brief PLM intervention might not produce any further improvements. Thus we hypothesized that students who demonstrated less initial algebra mastery would have had more room for improvement in their algebra skills and therefore were predicted to have been more likely to show effects of perceptual learning.

Methods

Participants

Students (n = 51, 9 males) in undergraduate psychology courses at the University of California, Los Angeles participated for course credit. Eleven additional participants were excluded because of i) experimenter procedural errors (6), ii) failure to take posttest (1), and iii) various forms of noncompliance (4), such as rapid, random responding during the psychophysical task or PLM or failure to wear needed corrective lenses.

Design

We used a pretest-posttest design. Participants in both Learning and Control conditions completed an equation-solving assessment and the psychophysical task at pretest and completed the psychophysical task in a posttest given 24 hours later. The Learning Group participated in an Algebraic Transformations PLM immediately following the pretest. The Control Group checked for test/retest effects in the psychophysical task. The experiment was web-delivered on computer workstations using a standard browser.

Algebra Equation-Solving Assessment

The point of the algebra equation-solving assessment was to evaluate students’ initial algebra ability. Participants solved for the variable in ten simple one-variable equations. One of the most complex equations was (5w)/3 = 8+2. Instructions specified that participants use decimal form and that they could use a pen and scratch paper, but no calculators.

Psychophysical Task

On each trial, participants viewed a briefly presented pair of equations and made a forced choice of whether they were physically identical (“match”) or whether they differed in any way (“mismatch”). We used the method of constant stimuli, testing 24 trials at each of 5 presentation durations that were spaced according to a geometric series (200, 336, 564, 948 and 1593 ms). The lowest and highest values were aimed to capture presentation times at which performance was expected to be at or near chance, and at or near ceiling, respectively.

Equation pairs were presented in a single line of text with blank space between them. The equations were presented in black text on a white rectangle. Viewed from 3 feet away, equations had visual angles of 4.1 deg in width and 36 min in height, with a space between equations of 1.6 deg. At the onset of each trial, participants viewed a white fixation cross (1.5 deg) centered on a black background in the center of the screen for one second. Participants were shown labeled response keys for match (‘z’) and mismatch (‘>‘). Equations appeared where the fixation cross had been. Participants could respond as soon as they felt ready. At the end of the presentation duration, equations were replaced by a random-dot mask sized to just cover the equation rectangle. Once participants responded, the next trial began immediately or the participant was shown an end-of-block screen.

To create equation pairs, we generated three-term linear equations involving addition or subtraction, e.g. 13+27=5m. From each of these, we created systematic variations (not mathematical transformations) that were visually similar, e.g. 13=27+5m. From each original equation and its variations, we created four match and four mismatch pairs. Pairs were randomly assigned to blocks of eight, such that blocks had four match (and four mismatch) pairs with only one pair from an original equation.

For each participant at each test time, fifteen of the 32 blocks were randomly selected, ordered, and assigned presentation durations. Order of pairs (trials) within each block was also randomized. Trials in a block had the same duration, and three blocks were assigned to each duration, producing 24 trials per presentation duration. On rare occasions, a program error caused a trial to be repeated within a testing session; these repeated trials were removed from the data set (approximately 0.2% of trials).

PLM Intervention

Learning Group participants used a current version of the Algebraic Transformations PLM previously studied by Kellman et al. (2010). Most trials consisted of equation-mapping trials, in which a target equation was shown at the top of the screen and participants were asked to select the legal algebraic transformation of that target from four options: three distracters and one legal transformation, as in Figure 1. There were eight subtypes within equation-mapping trials, including easy and hard questions involving each of the arithmetic transformations: addition, subtraction, multiplication, and division. Equation mapping trials were always followed by simple feedback, indicating whether the participant’s answer was correct or incorrect, and highlighting the correct answer as needed.

Figure 1:

Screen shot of an equation-mapping trial in the Algebraic Transformations PLM, showing feedback.

It is important to note that no trials in the PLM resembled the trials in the psychophysical task trials; in other words, the PLM provided no practice in deciding on a physical match of two equations under conditions of restricted presentation time; thus, effects of PLM use on the psychophysical task would be transfer effects, providing insight into deeper encoding changes produced by the PLM.

PLM use continued for each participant until he or she achieved objective mastery criteria; these combined accuracy across successive trials, under a criterion response time for each type of transformation.

Procedures

Participants were randomly assigned to either the Learning Group or the Control Group. On the first day of the study, both groups were given the same pretest: the equation-solving assessment followed by the psychophysical equation match-mismatch task. Control Group participants were released. Immediately after the pretest, the Learning Group worked on the PLM until reaching termination criteria, before being released. All participants returned the next day for the posttest.

Psychophysical Task

Participants read instructions and continued through the five practice trials and all fifteen blocks of the match-mismatch task. On any trial, if the participant did not respond within 2 seconds of the mask onset, then text appeared prompting participants to respond.

PLM Intervention

Learning Group participants remained after the pretests for the PLM intervention. They were given a brief introduction to the PLM. Participants were informed that their accuracy and reaction time would be tracked, so random guessing would not help them complete the PLM.

Once participants achieved mastery criteria, they saw a congratulatory screen and were released. Participants who did not reach mastery were released at the end of a total of two hours of pretesting and PLM participation. All but 3 participants completed the module to mastery.

Posttest

The psychophysical (match mismatch) task was administered again, one day later.

Dependent Measures and Analyses

Equation-Solving Assessment

We measured accuracy in terms of score (number correct trials out of 10), and response time on correct trials for the equation-solving assessment. Performance on this assessment was used to split participants into High- and Low-Ability groups for ability-based analyses on the psychophysical task.

Psychophysical Task

We analyzed proportion correct at each presentation duration for the psychophysical task at pretest and posttest in each group. (We also fit psychometric functions to these data using logistic functions with maximum likelihood fitting, but those analyses, while consistent with the data reported here, provided little additional information and are omitted here.)

We also conducted analyses for each ability group, to test our hypothesis about how students with lower initial algebra ability would show stronger PL effects.

We planned comparisons of i) Learning Group pretest and posttest accuracy at each duration and ii) change in accuracy across conditions (Learning versus Control) using difference scores (accuracy change = posttest accuracy – pretest accuracy). T-tests were all two-tailed.

PLM Intervention

The PLM program recorded accuracies and reaction times for each trial, total time and total number of trials.

Results

Equation-Solving Assessment

We gave the equation-solving assessment to identify participants more likely to show learning effects. Participants were split into High- and Low-Ability Groups based on a median split regardless of condition assignment. This procedure yielded a Low-Ability Group of 26 participants (n_Learning=11, n_Control=15) and a High-Ability Group of 25 participants (n_Learning=14, n_Control=11).

PLM Intervention

Overall, the Learning Group spent an average of 30.2 minutes (median = 21.4 minutes) on an average of 141.9 trials (median = 110 trials) in the PLM. As predicted, ability groups differed on both time and number of trials in the PLM, such that the Low-Ability Group required more practice on the PLM to reach mastery than the High-Ability Group: the Low-Ability Group ($\overline{x}=46.0$ minutes) spent a significantly longer time training on the PLM than the High-Ability Group ($\overline{x}=17.8$ minutes), F(1,23)=11.04, p=.003, η_p²=.324. The Low-Ability Group ($\overline{x}=180.9$) also required significantly more trials than the High-Ability Group ($\overline{x}=105.8$), F(1,23)=9.88, p=.005, η_p²=.300. Three students did not reach mastery; on average they retired about half of the categories, spending an average of 87.5 minutes 325.7 trials. All three were in the Low-Ability Group. Comparing ability groups on the PLM without these three reduces the time and trial differences between groups, but the differences and are still reliable. These participants were included in further analyses.

Psychophysical Task

We tested durations aimed at capturing a full range of accuracies from floor to ceiling because we did not know a priori in what part of the range we might find PL effects. Results showed that the shortest (200 ms) and longest (1593 ms) presentation durations were effective at capturing the low and high ends of the range, such that they showed clear floor and ceiling effects. The shortest presentation duration of 200 ms showed a floor effect or chance accuracy (pretest proportion correct = .51, posttest = .54). The longest presentation duration, 1593 ms, produced pretest accuracy of .87 and posttest accuracy of .88. Consistent with occasional accidental key presses and lapses in attention, these values were considered to be at or near a theoretical ceiling of about .90 accuracy.

Further analyses focused on the middle three presentation times. Figure 2 illustrates the full results. The Learning Group improved more on the match-mismatch task from pretest to posttest than the Control Group at longer presentation durations. A condition (Control, Learning) x test time (pretest, posttest) x presentation duration (336 ms, 564 ms, 948 ms) repeated-measures ANOVA on raw accuracy confirmed a trending¹ 3-way interaction, Pillai’s trace F(2,48)=3.149, p=.052. There were also main effects of test time, F(1,49)= 14.091, p<.001, η_p²=.223, and presentation duration F(1,49)=94.021, p<.001, η_p²=.797. There were no other reliable main effects or interactions.

Figure 2:

Average accuracy as a function of presentation duration (x-axis), test time (bars), and condition (panels). Error bars indicate standard errors of the mean.

Planned comparisons at each presentation duration showed that the Learning Group was more accurate at posttest than pretest, and this difference was reliable at the 948 ms duration, t(25)=−4.122, p<.001. At 948 ms, the Learning Group had a significantly greater increase in accuracy than the Control Group, p=.046. No other planned comparisons had reliable results.

Low-Ability Group

Splitting students into Low-Ability and High-Ability Groups based on their equation-solving assessment revealed that almost all the improvements in encoding occurred in the Low-Ability Group, as shown in Figure 3. The Low-Ability participants in the Learning Group showed reliably greater increases in accuracy than Low-Ability participants in the Control Group, especially for longer presentation durations. This pattern of results was confirmed by a two-way repeated-measures ANOVA of condition and presentation duration on accuracy change (difference scores). There was a significant two-way interaction, F(2,23)=5.29, p=.013, η_p²=.315, such that at 948ms the Learning Group increased their accuracy while the Control Group did not, and performance was not different across groups at the other durations. There were no reliable main effects.

Figure 3:

Average accuracy change as a function of presentation duration (x-axis), condition (bars), and ability group (panels) using difference scores. Error bars indicate ± one standard error of the mean.

Planned comparisons of Learning Group accuracy at pretest and posttest revealed a significant increase in accuracy at 948 ms, t(10)=−3.06, p=.012. Comparisons of accuracy change confirmed that at 948 ms the Learning Group increased their accuracy significantly more than the Control Group, t(24)=2.70, p=.011. Other planned comparisons were not reliable.

High-Ability Group

High-Ability participants in the Learning Group showed no benefit of the PLM intervention relative to the Control Group. This was confirmed by a two-way repeated-measures ANOVA of condition and presentation duration on accuracy change. There were no reliable main effects or interactions.

Planned comparisons of Learning Group pretest and posttest accuracy revealed a significant gain in accuracy at 948 ms (${\overline{x}}_{\text{pre}}=.76$, SE_pre=.03; ${\overline{x}}_{\text{post}}=.83$, SE_post=.02), t(13)=−2.709, p=.018, but there were no reliable differences in accuracy change across conditions (Learning, Control).

The different pattern of results for each ability group cannot be explained by preexisting differences on the psychophysical task: a three-way ANOVA of ability group and condition and presentation duration on pretest accuracy revealed no reliable main effects or interactions involving ability group or condition.

Discussion

College students demonstrated significantly improved encoding of mathematical objects at a 24-hour delay after a brief perceptual learning intervention. The same level of improvement was not shown by a Control Group. These results indicate that even a relatively brief PL intervention can lead to durable changes in basic information extraction, detectable using psychophysical methods.

These results are remarkable, in several ways. One is that these results appear to be the first evidence of specific encoding changes produced by the use of a high-level mathematics learning intervention. After use of the PLM, learners showed encoding improvements that were manifest in the initial second of contact with new mathematical expressions. We found these effects using a psychophysical task in which participants simply judged whether or not two briefly presented equations were physically identical. This task was quite different from mapping transformations, over much longer time periods, that participants performed in the PLM. Second, we detected these changes after a 24-hour delay, indicating that these are not transient effects.

Conceivably, resulting changes in information encoding could have been at other levels, such as in discrimination of elementary symbols or characters, or in more complex perceptual recognition of shifting or alteration of specific terms in algebraic transformations. Indeed, there may also be encoding improvements on tasks such as these. The current results, however, show that PLM interventions in math induce at least some basic, durable encoding changes.

These initial findings are also striking in that undergraduate participants all had extensive previous algebra exposure and competence. All took college entrance examinations and were admitted in a highly competitive admissions process. On the SAT Reasoning Test, freshmen who enrolled at UCLA in the fall of 2013 had an average Math Section score of 654 out of 800 (“Profile of Admitted Freshmen” – UCLA, 2013). 74% of freshmen scored at least 600 – at or above the 74^th percentile of test takers (The College Board, 2014). Despite substantial experience with algebra and higher mathematics, a mere 30 minutes on the PLM significantly increased their accuracy in extracting equation structure in a relatively enduring way.

The results in this study were modest for the Learning Group as a whole but robust for students who showed lower algebra proficiency at pretest. This pattern suggests that students with high initial algebra performance were already at or near mastery, at least as defined by the learning criteria in the Algebraic Transformations PLM (Kellman et al., 2010); they required little learning or practice to complete the PLM and thus did not show the learning effect. The results for the lower proficiency participants, however, indicates that a PLM intervention designed for middle or early high school students may improve basic encoding even among university students. It is likely that even greater PL-induced changes in basic information extraction may be detectable in younger or less proficient students. This is an important topic for future research.

The application of psychophysical methods to complex cognitive domains, such as mathematics, seems unusual, even anomalous. Searching for, and finding, changes in information encoding in mathematics learning, for example, may be understood in terms of important connections among perception, PL, cognition, and learning (Kellman & Massey, 2013), but more typically perception and complex cognitive tasks have often been considered to have little relation. As a variety of recent work suggests, perception of relational structure, and its improvement through PL, is a primary component of learning and expertise in high-level domains, moreso than has been generally recognized in research or implemented in instruction (Kellman & Massey, 2013). In future work, combining psychophysical and new instructional methods may lead to revealing synergies in understanding and optimizing mechanisms of learning.

In sum, we found direct evidence of durable encoding changes due to PL in mathematics: perceptual learning increases the accuracy of speeded equation comparisons. PL interventions have already shown strong benefits in notoriously hard parts of mathematics learning (Kellman et al., 2010; Massey, Kellman, Roth, & Burke, 2011), as well as in other domains (Kellman, 2013; Krasne et al., in press). The present results open a door to a more detailed understanding of the aspects of learning, even in complex, symbolic tasks, that advance via attunements and improvements in the pickup of information. Progress in exploring these components of learning, and their relations to equally important declarative and procedural aspects of learning, may offer great potential for addressing chronic problems in STEM learning and revealing missing links in theory and instruction.