Using Wise Interventions to Motivate Deliberate Practice

Abstract

Deliberate practice leads to world-class excellence across domains. In the current investigation, we examined whether psychologically “wise” interventions targeting expectancies and values—stock antecedents of ordinary effortful behaviors—could motivate non-experts to engage in deliberate practice and improve their achievement. As a preliminary, we developed and validated a novel task measure of deliberate practice and confirmed its association with (a) expectancy-value beliefs and (b) achievement in the non-expert setting (Study 1). Next, across four longitudinal, randomized-controlled, field experiments, we intervened. Among lower-achievers, wise deliberate practice interventions improved math performance for fifth and sixth graders (Study 2), end-of-semester grades for undergraduates (Study 3), and end-of-quarter grades for sixth graders (Study 4); the same pattern of results emerged in end-of-quarter grades for seventh graders (Study 5). Following the intervention, expectancy-value beliefs and deliberate practice improved for one month (Study 4), but not four (Study 5). Treatment proved beneficial over and above two control conditions, one that taught standard study skills (Studies 2 and 3), and one that discussed deep interests, generalized motivation, and high achievement (Studies 4 and 5). Collectively, these findings provide preliminary support for the heretofore untested hypothesis that deliberate practice submits to the same laws that govern typical forms of effortful behavior, and that wise interventions that tap into these laws can spur short-term gains in adaptive beliefs, deliberate practice, and objectively-measured achievement.

Deliberate practice—a challenging yet highly effective form of practice—leads to world-class eminence across domains. It helps turn chess players into grandmasters, violinists into virtuosos, and gymnasts into Olympians (Ericsson, 2006, 2008). Even critics, who point to other relevant determinants of skill improvement, including talent, acknowledge deliberate practice is an “unquestionably important” predictor of success (Macnamara, Hambrick, & Oswald, 2014, p. 1615).

If deliberate practice is so effective at improving performance, why isn’t it more commonly taught? Can non-experts be motivated to practice deliberately? Across four longitudinal, randomized-controlled, field experiments, we examined whether interventions that targeted expectancies and values, two established antecedents of ordinary effortful behaviors, could motivate deliberate practice among non-experts and improve achievement over time. Our intervention built on foundational research which documents that brief, psychologically “wise” interventions—theoretically-informed interventions that “take as their primary target a change in a specific psychological process” (Walton, 2014, p. 73)—can generate surprisingly large and durable effects (Garcia & Cohen, 2012; Walton, 2014; Wilson, 2011; Yeager & Walton, 2011).

Deliberate Practice and Achievement

In deliberate practice, one engages in tasks “that are initially outside [one’s] current realm of reliable performance, yet can be mastered within hours of practice by concentrating on critical aspects and by gradually refining performance through repetitions after feedback” (Ericsson, 2006, p. 692). Deliberate practice has four main components: (1) the individual must have a well-defined goal for improving a specific aspect of performance, (2) the level of challenge should just exceed the individual’s skill, (3) the individual receives immediate feedback, and (4) the individual repetitively focuses on the correction of error. Deliberate practice is associated with high levels of accomplishment in ballet, computer programming, aviation, firefighting, music, medicine, sports, and sales, among other fields (Ericsson, 2006, 2008). Most pertinent to the current investigation, undergraduates who do deliberate practice when they study earn higher GPAs (Plant, Ericsson, Hill, & Asberg, 2005).

There have been only a few attempts to increase deliberate practice via intervention. In these studies, conducted almost exclusively in the medical domain, clinicians are randomized to learn a medical procedure via deliberate practice (e.g., computerized simulators that offer immediate feedback and opportunities for repetitive practice) or training as usual (e.g., clinical rounds) (Kessler et al., 2011). A recent meta-analysis (total N = 633) confirms that deliberate practice leads to faster acquisition of medical skills than traditional training, d = .71, p < .001 (McGaghie et al., 2011). To our knowledge, only one study has assessed a deliberate practice intervention in a non-medical, academic setting: Undergraduate physics students randomized to learn about electromagnetism via deliberate practice (e.g., they were given pre-class reading quizzes, in-class clicker questions, and other activities designed to identify weaknesses and provide feedback) scored higher on a follow-up assessment than peers who received traditional lectures (Deslauriers, Schelew, & Wieman, 2011).

These past interventions demonstrate that when curricula are re-engineered to require deliberate practice, faster learning ensues. In the present investigation, we address a novel question: Can deliberate practice be motivated? Can an intervention spur individuals to freely initiate and sustain deliberate practice? Whereas correlational, longitudinal research documents that individuals who engage in more deliberate practice achieve to higher levels than their peers (Ericsson, 2006), and experimental research demonstrates that redesigning a course can promote deliberate practice, to date, next to nothing is known about whether deliberate practice can be intentionally motivated.

Motivating Deliberate Practice

Deliberate practice, is “…[not] for the faint of heart” (Ericsson, Prietula, & Cokely, 2007). According to dancer Martha Graham (1998), practice involves “times of complete frustration … daily small deaths” (p. 95). Across domains, most individuals report that deliberate practice is, if not deadly, effortful and unpleasant (Duckworth et al., 2011; Ericsson, 2006, 2007, 2009; Ericsson & Ward, 2007; Ericsson, Krampe, & Tesch-Romer, 1993). Performers in all domains, including sports, find that mental exhaustion limits how much deliberate practice they complete (Ericsson, 2002, 2003). Even seasoned experts sustain, on average, no more than four hours of deliberate practice per day (Ericsson, 1996; Ericsson et al., 1993). Long before academic research on deliberate practice began, Bryan and Harter (1897) concluded that telegraph operators plateaued in their skill level because they would “not make the painful effort necessary to become experts” (p. 51).

What motivates some to undertake these “painful efforts”? Two related psychological antecedents of deliberate practice have been identified: grit (Duckworth et al., 2011) and harmonious passion (Bonneville-Roussy, Lavigne, & Vallerand 2011; Vallerand et al., 2007; Vallerand et al., 2008). Grit is the tendency to sustain effort and interest towards a single pursuit over time (Duckworth et al., 2007). In a study of National Spelling Bee finalists, hours of deliberate practice fully mediated the prospective association between grit and final competition performance (Duckworth et al., 2011). Similarly, individuals with harmonious passion—those who have autonomously internalized cherished activities into their identities—do more deliberate practice, and this explains their higher levels of achievement in basketball, ballet, music, and other domains (Bonneville-Roussy, Lavigne, & Vallerand 2011; Vallerand et al., 2007; Vallerand et al., 2008).

Building on this correlational research, we conducted an experimental test of a causal hypothesis: Could targeting established determinants of effortful behavior motivate deliberate practice? We moved away from expert, high-achieving populations to work with non-expert students who, as a group, do not exhibit unusual motivation to invest effort in deliberate practice in any domain. In doing so, we address the clarion call of Ericsson and Charness (1994), who urged scientists to attain “A better understanding of … factors that motivate and sustain … deliberate practice” (p. 745).

An Expectancy-Value approach to intervention

To motivate deliberate practice, our intervention targeted expectancy and value, two key determinants of effortful behavior (Atkinson, 1957; Battle, 1965; Crandall, 1969; Eccles & Wigfield, 2002; Feather, 1982; Pintrich, 2003; Wigfield & Eccles, 2000). According to Expectancy-Value Theory (EVT), expectancy is the extent to which people believe they will succeed. Value, which refers to the subjective value the individual attaches to success, varies positively with perceived benefits (e.g., attainment value, intrinsic value, utility value) and inversely with perceived costs (e.g., the opportunity cost, emotional cost, and/or required effort). Expectancy and value are reliably associated with effort expenditure and achievement among non-experts (Eccles & Wigfield, 2002; Wigfield & Eccles, 2000), particularly in academic settings (Eccles, Midgley, & Adler, 1984; Eccles, Wigfield, Harold, & Blumenfeld, 1993; Meece, Wigfield, & Eccles, 1990; Nagengast, Trautwein, Kelava, & Ludtke, 2013).

Interventions that target expectancy and/or value have been shown to improve academic performance (Hulleman, Barron, Kosovich, & Lazowski, in press). For example, growth mindset interventions, which teach students that the brain can grow with effort, improve end-of-year grades, particularly for lower-achievers (Aronson, Fried, & Good, 2002; Blackwell et al., 2007; Good, Aronson, & Inzlicht, 2003; Romero, Paunesku, & Dweck, 2011; for review, see Yeager & Dweck, 2012). Similarly, interventions that target perceived costs, a component of value in EVT, improve academic achievement. In one study involving sixth graders, students exposed to a 10-minute intervention that encouraged them to reappraise learning costs (e.g., difficulty during learning) as normative outperformed controls on a follow-up reading comprehension task (Autin et al., 2012). Likewise, undergraduates taught to reappraise the anxiety experienced during high-stakes exams as a good sign scored higher on the GRE several months later (Jamieson et al., 2010).

The present intervention paralleled past interventions by targeting expectancies and costs, yet placed a novel focus on the role of practice. For example, in addressing expectancy, the intervention taught: “Many people think talent is all that matters … actually scientific evidence suggests that deliberate practice is incredibly important to improvement and success.” This message is distinct from the focal message of growth mindset interventions, which teaches students that if they invest effort, their brains will grow. We believed it would be important to instill this practice-specific expectancy given the prevalence of the opposing belief—that talent, not practice, determines success (Tsay & Banaji, 2011). The intervention also encouraged students to reappraise costs specific to practice: “If you are frustrated or confused while practicing … it can mean you are working on your weaknesses … when you practice and everything goes perfectly, it may feel good, but it’s probably a sign that you’re not challenging yourself.” We thought it would be crucial to introduce this reappraisal because expending effort is inherently aversive and, all other things being equal, avoided (Eisenberger, 1992; Gray, 2000; Kool, McGuire, Rosen & Botvinik, 2010; Smith and Walker, 1993). Our focus on practice-related costs differentiated the present intervention from past interventions that have encouraged students to reappraise either learning as a whole (Autin et al., 2012) or the specific anxiety experienced during exams (Jamieson et al., 2010).

Intervening wisely

While it may seem surprising to suggest that a brief intervention could affect academic achievement over time, psychologically “wise” interventions have been shown to produce surprisingly large and durable effects on effort expenditure and achievement, among other outcomes (Blackwell et al, 2007; Walton & Cohen, 2011; Aronson, Fried, & Good, 2002; Cohen et al., 2009; Hulleman & Harackiewicz, 2009; for reviews, see Garcia & Cohen, 2012; Yeager & Walton, 2011). How do they work? Wise interventions target key psychological processes that influence outcomes of interest (Lewin, 1952; Ross & Nisbett, 1991). Targeting such processes can generate initial benefits and, through recursive processes, perpetuate themselves long-term. For example, a student who comes to believe more strongly in the importance of effort might study harder and perform better in school. Improved performance may then reinforce the belief that effort matters, jumpstarting the cycle anew. In the current investigation we designed a brief, wise intervention in the hopes of changing deliberate practice habits and achievement over time.

Helping the least expert

We expected the intervention would improve beliefs and deliberate practice behavior for all students, but preferentially boost the achievement of lower-performers—the least expert of the non-experts. Wise interventions often differentially improve the performance of lower-achieving students (Cohen et al., 2009; Hulleman & Harackiewicz, 2009; Paunesku et al., 2015; Wilson & Linville, 1982; Yeager et al., 2014a). This may be due to ceiling effects: Students already earning A’s cannot improve their grades, but students earning D’s, C’s, or B’s have room to grow. Because most students in non-expert settings do less deliberate practice than they are capable of, we believe almost all students could benefit from learning about deliberate practice. However, given that there is an artificial ceiling on measures of achievement in the academic context (e.g., GPA), we expected the intervention would preferentially raise the grades of lower-performing students.

The Current Investigation

Across four longitudinal, randomized-controlled, field experiments, we examined the effect of a wise intervention on expectancy-value beliefs, deliberate practice, and achievement. As a preliminary (Study 1), we validated a novel behavioral measure of deliberate practice and examined the relationship between deliberate practice and achievement in the non-expert middle school setting. Using this task, we also confirmed associations between deliberate practice and expectancy-value beliefs. Next, we intervened. We tested the effect of a wise, 25-minute intervention on middle school math performance (Study 2) and college grades (Study 3). Finally, we developed an expanded 50-minute intervention for middle school students—sixth (Study 4) and seventh (Study 5) graders—and evaluated its effects on academic achievement, self-reported beliefs, and deliberate practice behavior, measured at one-month (Study 4) and four-month (Study 5) follow-up. Whereas in Studies 2 and 3 the control condition instructed students in study skills, in Studies 4 and 5 it discussed interests and achievement. This combination of studies allowed us to test the efficacy of the intervention across a range of students (aged middle school through college) and a range of outcomes (middle school math performance to end-of-semester grades to end-of-quarter grades).

Since each wise intervention we administered was both informational (e.g., an introduction to the tenets of deliberate practice) and motivational (e.g., content more explicitly aimed at changing self-relevant expectancies and values associated with practice), in Study 5, we introduced an additional condition in which students received information about deliberate practice without an accompanying motivational lesson. With this hybrid condition, we parsed the intervention’s informational effects from its motivational ones to isolate active ingredients.

Study 1

In Study 1, we assessed the importance of deliberate practice to non-expert achievement and examined the association between expectancy-value beliefs and deliberate practice in the middle school setting. Diary measures and retrospective self-report measures are traditionally used to measure deliberate practice (Ericsson et al., 1993; for review, see Ericsson, 2006); however, we doubted whether average middle school students would be able to reliably distinguish between deliberate practice and less effective forms of practice, a distinction existing self-report measures rely on. Indeed, our intervention approach is premised on the assumption that deliberate practice is not widely understood among non-experts. Therefore we developed a novel, objective task measure of deliberate practice.

In the Deliberate Practice Task (DPT), students work through a series of challenging online math problems in an online browser during a 45-minute class period. At any point, students may switch away from the math task to YouTube, Facebook, Instagram, or other recreational websites by opening new tabs. The math task captures several important features of deliberate practice: It keeps students in a zone of challenge by leveling them up to harder problems when they answer easier ones; it offers students the opportunity to view immediate feedback (e.g., worked out answers with explanations); finally, it encourages repetition by presenting iterations of similar problems until mastery is achieved. Because the DPT captures key features of deliberate practice, deliberate practice was operationalized as the amount of time students chose to spend on-task (time focused), as opposed to off-task, during a 45-minute class period.

We first examined whether the DPT demonstrated convergent validity with Big Five conscientiousness and its facets, grit and self-control. We also tested whether the DPT demonstrated discriminant validity from dissimilar constructs, such as Big Five extraversion, agreeableness, openness, and emotional stability. Next, we examined the incremental predictive validity of the DPT for GPA, an objective measure of academic achievement. Finally, we assessed the relation between the DPT and two hypothesized psychological antecedents of deliberate practice: expectancies and values.

Method

Participants and procedure

Participants were sixth (n = 542) and seventh (n = 417) graders attending two school districts in the U.S. Approximately 68% were White, 15.3% Asian, 8.5% Hispanic, 8.2% Black; 51.1% were female. Seven percent of students qualified for free or reduced price meals, an indication of low household income.

Students completed all activities on school computers over three consecutive days. On Days 1 and 2 students completed self-report measures; on Day 3 students completed the DPT. During all research sessions, students wore ear buds and each computer was covered with a privacy screen—a plastic sheet that lessens visibility to anyone not seated directly in front of the computer.

Because some students did not complete some assessments, degrees of freedom varied by approximately 8% across analyses. Missing data were due to students opting out, technological malfunctions (e.g., computer crashing), and absences. In addition, we excluded DPT data from participants who spent less than 5 minutes signed into the task (approximately 2% of the sample). Short task time may have resulted from early dismissal, students accidentally closing the browser, students intentionally closing the browser, a technological glitch, or other causes. Pairwise deletion was used across analyses. In separate models (not shown), listwise deletion yielded virtually identical results. For more information on the final sample, see the online supplement.

Measures

Deliberate practice task (DPT)

The DPT included standard textbook math problems (e.g., 6x = 10) at nine skill levels. Math problems and solutions were taken from Khan Academy, a free, educational website (www.khanacademy.org). The task was tested with several students beforehand to ensure math content was age-appropriate. The task started all students with math problems at the easiest level, Level 1. After each set of five problems, students received their score (number correct out of 5) and had the opportunity to view worked-out solutions to the problems they had completed. Students who answered five consecutive problems correctly were advanced to the next level. The aim of this leveling up feature was to keep students in a zone of constant challenge.

Before beginning the math task, students were told to treat the task like a regular school assignment. They were also told, “Because today’s math lesson is hard, you are allowed to do something you’re not usually allowed to do in school: take breaks on the internet.” In coordination with school administrators, recreational websites (e.g., Youtube, Facebook, Instagram) were unblocked for the entire 45-minute period, allowing students to open additional browser tabs and visit sites of their choosing.

Time focused

The amount of time students focused on math problems was measured using an embedded timer invisible to the user. This timer monitored the number of seconds students spent on the tab with math problems versus other tabs. Because the task approximated key features of deliberate practice (e.g., it leveled students up, it provided them with the opportunity to view feedback, it allowed for repetition), we operationalized deliberate practice as the amount of time students focused on the math task during the 45-minute period.

Total task time

This was the total number of seconds that students were signed into the task during the 45-minute class period, regardless of whether they were working on math (time focused) or surfing the internet.

Self-report questionnaires

Students rated all questionnaire items on a 5-point Likert-type scale from 1 = not at all like me to 5 = very much like me, except where otherwise noted. Items for each scale were averaged to create a composite. Higher values corresponded to greater endorsement of the specified construct. We included two types of expectancy-value scales: (1) practice-specific scales (deliberate practice beliefs, frustration tolerance during practice), which narrowly assessed an individual’s expectancies and values related to practice, and (2) non-practice-specific scales, which assessed expectancies and values that were not directly related to practice (e.g., self-efficacy, locus of control, distress tolerance). For descriptive statistics, see Table 1.

Table 1.

Descriptive statistics and partial correlations (pr) with time focused for Study 1

	M	SD	n	pr
Deliberate practice task
Time focused (seconds)	1127	378	885	—
Total task time (seconds)	1448	305	885	—
Big Five Personality
Big Five conscientiousness	3.91	0.77	831	.22***
Grit	3.45	0.66	843	.21***
Self-control	3.31	0.69	841	.20***
Big Five agreeableness	4.08	0.65	832	.11**
Big Five emotional stability	2.86	1.01	832	.06
Big Five openness	3.98	0.71	833	−.02
Big Five extraversion	3.89	0.69	835	−.09**
Academic performance
GPA	88.39	7.80	885	.19***
Expectancy-value beliefs
Practice-specific
Deliberate practice beliefs	77.09	13.82	844	.14**
Frustration tolerance	3.67	0.68	844	.15***
Non-practice-specific
Growth mindset	3.60	0.86	844	.03
Locus of control	3.97	0.57	844	.11**
Distress tolerance	2.56	1.05	844	−.06

Note.

^**p < .01,

^***p < .001.

All partial correlations control for school site, gender, ethnicity, standardized math test achievement, and total task time. Partial correlations with Big Five personality scales appear in descending order of magnitude.

Big Five personality

We adapted items from the Big Five Inventory to improve reading comprehension (John & Srivastava, 1999). We used four items per trait to assess conscientiousness (e.g., “I am organized and neat”) (α = .85), agreeableness (e.g., “I am considerate and kind to almost everyone”) (α = .73), emotional stability (e.g., “I get stressed out easily”; reverse-scored) (α = .76), openness to experience (e.g., “I am curious. I am very interested in learning new things”) (α = .65), and extraversion (e.g., “I am outgoing, I like tomeet new people”) (α = .64).

Grit

Students completed the 8-item (e.g., “I am a hard worker”) Short Grit Scale (Duckworth & Quinn, 2009) (α = .78).

Self-control

Students completed the 13-item Brief Self-Control Scale (e.g., “People would say that I have iron self-discipline”) (Tangney, Baumeister, & Boone, 2004) (α = .85).

Practice-specific expectancy and value

To assess practice-specific expectancies, students completed a measure of Deliberate Practice Beliefs. In this measure, they were asked to “advise another student on how to succeed in school” by rating each of the five main components of deliberate practice (e.g., focus on weaknesses, concentrate 100%) using sliders that ranged from 0 = not at all helpful, to 100 = very helpful. Due to time constraints, this measure was only administered in one of the two school districts. To assess costs, one component of value in EVT, students answered a 3-item measure of frustration tolerance (e.g., “If you are frustrated and struggling when practicing, it is a sign that you are challenging yourself and improving”), which measured the degree to which participants endorsed frustration during practice as a sign of learning (α = .44). Removing the one non-reverse scored item from the frustration tolerance scale improved its alpha reliability (α = .63). Re-running analyses with this one item removed led to no significant changes in the reported results, so the full 3-item scale was used in all reported analyses.

Non-practice-specific expectancy and value

To assess non-practice-specific expectancy, participants completed two standard, less specific measures of expectancy: the 3-item (e.g., “You have a certain amount of intelligence and you cannot change it”, reverse scored) Growth Mindset Scale (Dweck, Chiu, & Hong, 1995) (α = .72) and the 4-item (e.g., “If your teacher gives you a bad grade it’s not your fault”, reverse scored) Locus of Control Scale (Wellborn, Connell, & Skinner, 1989) (α = .46). To assess non-practice-specific costs, participants completed three items (e.g., “Feeling upset is unbearable to me”, reverse scored) that assessed distress tolerance (α = .82).

Grade point average (GPA), standardized math achievement test scores, and demographics

From school records, we obtained first-quarter GPA for all core subjects (English, social studies, science, math), standardized math achievement test scores from the prior year, gender, and ethnicity for all students. Because measures of achievement were on different scales at different school sites, these measures were standardized within schools prior to analyses.

Results and Discussion

Preliminary analyses

On average, students correctly solved only 22% of the math problem sets they attempted, suggesting that the task successfully kept students in a zone of challenge. Controlling for demographics (e.g., gender, ethnicity), school affiliation, and total survey time, students with higher standardized math achievement scores were more focused (pr = .13, p < .001) (for additional descriptives, see Table 1).

Next, we conducted a series of partial correlations to examine convergent, discriminant, and incremental predictive validity, as well as the association between deliberate practice and expectancy-value beliefs. Across analyses, we controlled for standardized math achievement test scores from the prior year, ethnicity, gender, school affiliation, and total task time. Results were not moderated by these covariates in this study or subsequent studies.

Convergent and discriminant validity

Time focused was associated with Big Five conscientiousness (pr = .22, p < .001), grit (pr = .21, p < .001), self-control (pr = .20, p < .001), agreeableness (pr = .11, p = .002), and inversely, with extraversion (pr = −.09, p = .01) (see Table 1). Time focused was unrelated to Big Five openness and emotional stability (prs = −.02, .06, ps = .55, .12). Because Big Five factors were inter-correlated (average r = .19), we fit a simultaneous regression model with all the Big Five factors predicting time focused. In this model, only conscientiousness (β = .21, p < .001) and extraversion (β = −.14, p < .001) continued to predict unique variance. The negative correlation between extraversion and time focused parallels past research in which extraverted students earn lower grades (Poropat, 2009).

Given the conceptual similarity between grit and conscientiousness, and the high correlation between these constructs in the present dataset (r = .73, p < .001), we fit a simultaneous regression model to see whether grit predicted incremental variance in the tendency to stay focused over and above conscientiousness. It did (grit: β = .09, p = .02; conscientiousness: β = .10, p = .01).

Criterion-related validity

Students who spent more time focused earned higher GPAs (pr = .18, p < .001). This association did not change when math grades were excluded from the GPA calculation (pr = .18, p < .001), confirming that the DPT captured practice behaviors associated with achievement in general, not just math achievement.

Expectancy-value beliefs

Time focused correlated with practice-specific expectancies (deliberate practice beliefs, pr = .14, p < .01), and values (specifically, we assessed costs, one component of value in EVT; frustration tolerance, pr = .15, p < .001), but was not consistently associated with non-practice-specific expectancies (growth mindset, pr = .03, p = .36; locus of control, pr = .11, p < .01) or values (again, we assessed the cost component; distress tolerance, pr = −.06, p = .08).

Study 2

In Study 1, we validated a novel behavioral measure of deliberate practice, established the importance of deliberate practice to middle school achievement, and uncovered associations between expectancy-value beliefs—particularly those that were practice-specific—and deliberate practice. Building on these encouraging findings, in Study 2, we developed and administered a wise, 25-minute intervention to fifth and sixth graders. Students were randomized to either a treatment condition that aimed to motivate deliberate practice or an active control condition that taught standard study skills. Approximately one week later, all participants were instructed to master math concepts in Khan Academy (a free educational website). We evaluated whether this brief intervention improved math achievement.

Method

Participants

Students were fifth (n = 98) and sixth graders (n = 111) at four schools across the U.S. (N = 209). Approximately 53% of participants were White, 23.9% Black, 11.0% Hispanic, 9.1% Asian, and 5.3% were of other ethnicities; 44.0% were female.

Procedure

Overview

In the spring, participating schools administered a 25-minute online intervention to students. Because the intervention contained audio and video, students wore ear buds during all research sessions. In this study and all subsequent studies, an algorithm in Qualtrics, the online platform used to host the intervention, randomly assigned students to condition. Immediately prior to the intervention, participants completed a short math pretest.

During the second research session and all subsequent sessions, students were instructed to master multiplication and division content in Khan Academy. On average, the second research session took place five school days after the intervention. Schools allotted varying numbers of class periods to Khan Academy (anywhere from 5 to 9 class periods on consecutive school days) based on computer availability.

In this study and all subsequent studies, teachers were completely blind to condition assignment and intervention content. Teachers were not shown the intervention modules until the study was over, and were not informed in advance about the specific content being taught to students.

Intervention content

Treatment and control modules were made as parallel as possible. They were matched for amount of text, video length, number of images, and number of activities. Moreover, across treatment and control conditions, we employed two psychological tactics commonly used to make wise interventions “stickier”: descriptive norms and the saying-is-believing effect (E. Aronson, 1999; J. Aronson, Fried, & Good, 2002; Walton & Cohen, 2011; Yeager et al., 2014a; Yeager et al., 2014b). To incorporate descriptive norms, we showed participants anonymous quotes in which other students (who had participated in focus groups) described their practice habits and preferences. These quotes were edited by our research team for clarity and brevity. Sharing such quotes normalized the behaviors the intervention encouraged (Cialdini, Reno, & Kallegren, 1990; Goldstein, Cialdini, & Griskevicius, 2008; see Cialdini, 2003). Participants also completed “saying-is-believing” exercises, in which they wrote letters to other students advocating the importance of what they had learned. These exercises capitalize on the psychological insight that one of the most effective ways to persuade a participant of a message is to have the participant advocate the message to others.

Treatment condition

The treatment condition consisted of two parts. In Part I, students learned the tenets of deliberate practice: 1) Focus on weaknesses, 2) Get feedback, 3) Concentrate 100%, and 4) Repeat until mastery. Didactic slides were interspersed with activity prompts, an illustrative video, and a letter-writing exercise. The aim of Part I was to teach students about deliberate practice and how it differs from less effective forms of practice.

Part II was motivational. It targeted expectancies and values, two psychological variables highlighted by Expectancy-Value Theory. To address expectancies, the module taught that talent and effort both contribute to success, but that the relative importance of effort—especially effort invested in deliberate practice—is often underestimated. To address costs, a component of value in EVT, the module encouraged students to interpret frustration and confusion as positive signs that one is engaging in optimal practice activities. Thus, the intervention targeted practice-specific expectancies and costs in an attempt to motivate deliberate practice. For a fuller description of the procedure used to develop the intervention, the full intervention text, as well as a table showing how specific quotes from the intervention map onto each theoretical concept, refer to the online supplement.

As noted above, each lesson ended with a saying-is-believing exercise in which the participant wrote a letter to another student endorsing deliberate practice. For example, one treatment participant wrote:

Deliberate practice is studying weaknesses or something that you are struggling at. How I applied this is by doing flashcards on things I don’t know. At first I was using regular practice and everyone seemed so much better than me but when I started using deliberate practice my grades went up.

Control condition

Control participants were taught standard study advice. To rule out the possibility that any discussion of studying or practice would be as effective as teaching students about deliberate practice, controls learned how to manage their time (e.g., use a calendar or planner) and were given practical strategies for remembering information (e.g., mnemonics). Parallel to the treatment module, the control module featured short activity prompts and an illustrative video.

Like the treatment condition, the control condition featured descriptive norms, and ended with students writing a letter to another student, endorsing what they had learned. For example, one control participant wrote:

I have learned many new tricks for studying. I thought that the mnemonics were really cool. You can use the things you already know to make things or work easier. Mnemonics keep you hooked on them books. Next time I think I will use this skill to help myself to get through a hard problem. Studying is the key to success.

Measures

Khan Academy performance, math pretest, and basic demographics

Performance was quantified as the total number of points earned in Khan Academy. Students could earn up to 100 points in each of nineteen multiplication and division topic areas, for a maximum of 1900 points.

The math pretest had 3 to 5 multiplication and division problems (depending on the school’s time constraints) taken from Khan Academy. Khan Academy performance and pretest scores were standardized by site before being combined into a single variable. Before the math pretest, students self-reported gender and ethnicity.

Interest and engagement

At the end of the intervention, students rated how interesting and engaging they found the module. Students rated two adjectives (“interesting” and “engaging”) using 100-point slider scales. The two values (r = .72, p < .001) were averaged into a single composite.

Results and Discussion

Analytic plan for Studies 2 through 5

We analyzed effects using ordinary least squares regression. Regression models controlled for basic demographics (e.g., gender, ethnicity, school site) and available measures of baseline achievement (e.g., pretest scores, prior GPA, standardized test scores from the prior year). Because measures of achievement were on different scales at different school sites, achievement measures were standardized by site prior to analyses.

Since we expected the intervention to preferentially help lower-achievers, we tested for main effects and Condition X Prior achievement interactions. We probed all significant interactions using the Johnson-Neyman method (Hayes, 2013; Johnson & Neyman, 1936) to identify regions of significance. We ran analyses using R statistical software (available at http://www.r-project.org) as well as the SPSS PROCESS macro (Hayes, 2013).

Preliminary analyses

Treatment and control groups did not differ on baseline achievement or demographics, with the exception of baseline ethnicity (for more details, see online supplement). There was no between-condition difference in how interesting and engaging students found the modules, t(187) = −1.14, p = .26, d = .16, 95% CI [−.12, .45].

Khan Academy performance

Although the main effect of the intervention was not significant (M_treatment = 0.14, M_control = 0.05),¹ t(198) = 0.64, p = .52, d = .09,² 95% CI [−.18, .36] the intervention improved online math performance for lower-achievers, Condition X Math pretest interaction, b = −0.37, SE = .14, t(197) = −2.65, p = .01 (see Figure 1). A follow-up Johnson-Neyman analysis (Hayes, 2013; Johnson & Neyman, 1936) revealed that the effect of treatment was significantly positive among lower-performing students (<−0.76 SD below average on the pretest) and significantly negative among high-performing students (> 1.78 SD above average on the pretest). Although unexpected, the negative effect among high-performing students may have been due to the fact that the treatment condition advised students to focus on areas of challenge and weakness. For students who had already mastered the assigned math content (as evidenced by high pretest scores), Khan Academy performance did not constitute the sort of challenge the intervention encouraged them to confront. The intervention may have discouraged these students from working on problems that were too easy for them.

Figure 1.

In Study 2, the treatment condition improved math performance in Khan Academy for lower-achievers, and the control condition improved math performance in Khan Academy for higher-achievers. Shaded areas indicate regions of significance.

Study 3

In Study 2, a 25-minute intervention improved math performance for lower-performing middle school students. We next tested whether this effect would generalize to older students and an objective measure of academic achievement: college GPA.

Method

Participants

Participants were undergraduates at a liberal arts college (n = 60) and a research university (n = 60). Students were predominantly White (71.7%), female (69.2%), and in their first two years of college (74.2%).

Procedure

Overview

At the liberal arts college, students were recruited through an email sent to the entire student body (approximately 850 students) three weeks prior to finals. Participants were entered in a raffle to win a $100 Amazon gift certificate. A total of 63 students (approximately 7%) participated. Three students with no spring GPA on record were dropped from analyses. At the research university, the intervention was advertised on the psychology department’s website. Sixty students in a large introductory psychology course (14%) participated for course credit. Participation was open to students for two weeks prior to the start of reading days.

Intervention content

The intervention was nearly identical to the intervention administered in Study 2. Language was slightly modified to ensure it was age-appropriate, but content remained unchanged.

Measures

Academic achievement and basic demographics

Academic achievement was evaluated differently by site. The liberal arts college provided overall fall and spring semester GPAs, on 4-point scales. The research university released midterm and final grades for Psychology 001, on 100-point scales. Fall GPA (at the liberal arts college) and midterm scores (at the research university) were used as measures of prior achievement. Gender and ethnicity were also obtained from official school records.

Interest and engagement

We assessed interest and engagement using the same questions used in Study 2. As in Study 2, responses (r = .71, p < .001) were averaged into a single composite.

Results and Discussion

Preliminary analyses

Randomization was successful. Treatment and control participants did not differ on any baseline variables (for details, see online supplement). Nor did they differ in how interesting and engaging they found the modules, t(110) = −1.31, p = .19, d = .25, 95% CI [−.12, .62].

Academic achievement

The intervention had a main effect on end-of-semester grades (M_treatment 0.10 =, M_control = −0.19), t(111) = 2.03, p = .04, d = .38, 95% CI, [.01, .74]. This main effect was driven by lower-achievers: Condition X Prior achievement interaction, b = −.37, SE = .15, t(110) = −2.51, p = .01 (see Figure 2). A follow-up Johnson-Neyman analysis (Hayes, 2013; Johnson & Neyman, 1936) revealed that the effect of treatment was significantly positive among lower-performing students (< 0.08 SD above average on prior achievement).

Figure 2.

In Study 3, the intervention raised end-of-semester academic achievement for all undergraduates, and especially among lower-achievers. Shaded areas indicate regions of significance.

Study 4

In Studies 2 and 3, a brief psychological intervention improved academic performance among lower-achieving students in middle school (Study 2) and college (Study 3). In Study 4, we examined the psychological and behavioral processes the intervention activated. We used questionnaires to evaluate self-reported expectancies and values, and the Deliberate Practice Task (developed and validated in Study 1) to measure deliberate practice behavior. Expectancies, values, and deliberate practice were measured once pre-intervention and twice post-intervention. As in Study 1, we measured practice-specific as well as non-practice-specific expectancies and values.

In addition, we introduced an expanded intervention and a new control condition. Feedback from focus groups revealed that students were most engaged by interactive material. We therefore created a longer, 50-minute intervention that expanded on both Part I and Part II of the earlier intervention, by including more activities. In addition, to rule out the possibility that merely mentioning motivation and high achievement drove the treatment effect in Studies 2 and 3, control participants received a new control module that discussed motivation and achievement. In particular, students learned how deep interests develop and how deep interests might promote achievement in their own lives.

Finally, in Study 4, we tested the durability of the intervention’s effect on achievement. Whereas in Studies 2 and 3 we measured academic achievement several weeks later, in Study 4 we measured the intervention’s impact on achievement over an entire academic quarter (approximately 11 weeks later). Our timeline was informed by past studies which document that brief, wise interventions can generate positive effects on achievement that last months, and even years (Blackwell et al, 2007; Walton & Cohen, 2011; Aronson, Fried, & Good, 2002; Cohen et al., 2009; Hulleman & Harackiewicz, 2009; for reviews, see Garcia & Cohen, 2012; Yeager & Walton, 2011). One theory is that wise interventions generate long-term effects by targeting key psychological and behavioral processes in recursive settings, i.e., those in which small changes build over time (Walton, 2014; Yeager & Walton, 2011). Schools are inherently recursive environments: later lessons build on earlier content. Drawing on this foundational research, we tested the long-term effects of a brief intervention that targeted practice-specific beliefs (e.g., expectancies, costs) and deliberate practice.

Method

Participants

We invited sixth graders in a public school district in the U.S. to participate. Approximately 78% of students (N = 427) across three middle schools participated. Remaining students either opted out, experienced technological difficulties (e.g., computer crashed), or were absent. Participants did not differ from non-participants on any baseline variables (e.g., incoming GPA, gender, ethnicity). For further details, see the online supplement.

Approximately 65% percent of participants were White, 19.4% Asian, 7.5% Hispanic, 4.9% Black, and 2.8% were of other ethnicities; 48.7% were female. Approximately seven percent of students qualified for free or reduced price meals.

Procedure

Overview

First, students completed a battery of self-report scales and the Deliberate Practice Task (DPT) during three class periods on three consecutive days. Ten weeks later, at the end of the third quarter, students completed the intervention during two class periods on two consecutive days. One day after the intervention (Follow-up 1) and again one month later (Follow-up 2), students completed the same battery of measures they had taken pre-intervention (i.e., self-report scales and the Deliberate Practice Task). During all research sessions, students wore ear buds and privacy screens were attached to all computers. GPA was obtained at one time point pre-intervention (second quarter GPA) and at one time point post-intervention (fourth quarter GPA).

Intervention procedures and content

Treatment and control modules employed the same persuasive techniques (descriptive norms and the saying-is-believing effect) described in Study 2. Modules were carefully matched to contain the same-length text, same-length videos, and parallel activities. Across both conditions, didactic slides were interspersed with interactive multiple-choice questions, short-answer prompts, a self-quiz, and an illustrative video (for sample screenshot, see Figure 3).

Figure 3.

Screenshot of the intervention.

Treatment condition

As in Studies 2 and 3, the intervention had two parts. In Part I (25 minutes), students were taught the tenets of deliberate practice (refer to the description in Study 2 for more details). During focus groups, students preferred the term “deep practice” to “deliberate practice,” so “deep practice” was substituted. The aim of Part II (25 minutes) was to motivate students to use the deliberate practice techniques they learned about in Part I. As in the earlier intervention (Studies 2 and 3), Part II targeted expectancies and costs with the aim of encouraging students to reappraise practice-specific expectancies and values. Refer to the online supplement to see the full intervention text as well as a table which shows how specific quotes from the intervention map onto each theoretical concept.

Control condition

In Part I (25 minutes), students learned about deep interests and high achievement. They learned what it looks and feels like to be motivated, how to increase motivation, and how to work around the obstacles that stand in the way of motivation and high achievement (e.g., students were cautioned against relying on what their friends do and were told that instead, they should follow their own interests). The active motivational beliefs targeted in the treatment condition—expectancies and values—were not touched upon.

Building on Part I, Part II (25 minutes) discussed the link between deep interests, motivation, and success as well as what being motivated feels like. In exercises and activities that paralleled those in the treatment condition, students were encouraged to apply what they had learned to their own lives.

Measures

Self-report questionnaires

Recall that the wording of the intervention aimed to increase the expectancy that deliberate practice would lead to success and reduce the cost associated with deliberate practice. Because the intervention addressed expectancy and cost specifically as they relate to deliberate practice, we included practice-specific measures of expectancy (deliberate practice beliefs at Pre-intervention, α = .45; at Follow-up 1, α = .60; at Follow-up 2, α = .67) and cost (frustration tolerance at Pre-intervention, α = .38; at Follow-up 1, α = .63; at Follow-up 2, α = .56), as well as non-practice-specific measures of expectancy (growth mindset at Pre-intervention, α = .75; at Follow-up 1, α = .80; at Follow-up 2, α = .84; locus of control at Pre-intervention, α = .45; at Follow-up 1, α = .68; at Follow-up 2, α = .72) and cost (distress tolerance at Pre-intervention, α = .81; at Follow-up 1, α = .86; at Follow-up 2, α = .90). For full details on these measures, see Study 1.

Removing the single non-reverse scored item of the frustration tolerance scale improved its alpha reliability at pre-intervention (α = .54). However, because re-running analyses with this one item removed led to no significant changes in the reported results, the full 3-item scale was used in all reported analyses.

Deliberate practice task (DPT), time focused, and total task time

Participants completed the same task described in Study 1.

Grade point average (GPA), standardized test scores, and basic demographics

GPA, standardized achievement test scores from the prior year, gender, and ethnicity were obtained from school records. Because the intervention was administered in the last several weeks of the third quarter, we assessed the intervention’s effect on fourth quarter GPA, controlling for GPA from the prior quarter (second quarter).

Interest

Students rated how interesting they found each module on a 100-point slider scale. We averaged ratings for the two modules (r = .77, p < .001) to create a single measure of student interest.

Results and Discussion

Analytic plan for Studies 4 and 5

As in Studies 2 and 3, effects on GPA were analyzed using ordinary least squares regression (see Study 2 for further details on the analysis plan for regression). Gender, ethnicity, school site, standardized achievement test scores, and second quarter GPA were included as covariates in all models. Following precedent (e.g., Yeager et al., 2014b), we controlled for standardized achievement test scores and prior GPA to comprehensively account for incoming differences in student achievement. As in Studies 2 and 3, across Studies 4 and 5, achievement measures were standardized by site prior to analyses.

In contrast to GPA, beliefs and behaviors were assessed at multiple time points (once before and twice after the intervention). Changes in beliefs and behaviors were analyzed using linear mixed models in R statistical software (nlme package, v. 3.1). Missing data were considered missing at random (MAR). Parameters were estimated using restricted information maximum likelihood (REML) (Singer & Wilett, 2003). REML makes use of all available data and retains cases with partial missing data. Repeated Level 1 measures were nested within students at Level 2. Intervention condition, basic demographics (e.g., gender, ethnicity, school site), measures of incoming achievement (e.g., prior GPA, standardized achievement test scores from the prior year), and total task time (for DPT analyses only) were variables at the student level. Individuals were permitted to deviate randomly from the mean intercept. For more details on the linear mixed effects model, refer to the online supplement.

Paralleling analyses in Studies 2 and 3, we first analyzed whether the intervention had a main effect, and then whether the intervention preferentially changed outcomes (e.g., beliefs, behaviors, and academic achievement) among lower-performing students. The Johnson-Neyman method was used to identify regions of significance.

Because the intervention’s effects on beliefs and behaviors were not consistently moderated by prior achievement, and the intervention had no consistent effect on non-practice-specific beliefs, these analyses are not reported in the main text. For full details on these analyses, as well as figures documenting the intervention’s main effects on these outcomes, see the online supplement.

Data exclusions and missing data

In Study 4, as well as Study 5, many Deliberate Practice Task (DPT) sessions at Follow-up 1 were cancelled due to snow. Cancelled sessions were re-scheduled for the first day following the snow day, which was often a half-day. Students who took part in re-scheduled sessions experienced atypical testing conditions: On half-days periods were often shorter than usual, and at several school sites, internet connectivity was either stalled or non-functional during these re-scheduled sessions. In Study 4, the DPT session at Follow-up 1 was cancelled for approximately 40% of students. Because sub-optimal experiences with the DPT at Follow-up 1 (e.g., the internet stalling, altered period schedules) aggravated students and colored their associations with the task at Follow-up 2, DPT data collected following snow cancellations were excluded from analyses. This same protocol was applied in Study 5: DPT data collected following snow cancellations were removed from analyses.

Approximately 7% of students did not complete self-report questionnaires at each of the three administrations. Participants with missing or excluded data (e.g., DPT data, self-reported beliefs) did not differ systematically from participants without missing data on any baseline variables (e.g., gender, ethnicity, second quarter GPA). For further details, see the online supplement.

Preliminary analyses

Randomization was successful. Treatment and control groups did not differ on baseline variables (for more details, see online supplement). Likewise, there was no between-condition difference in how interesting participants found the modules, t(425) = 1.60, p = .11, d = −0.15, 95% CI [−.34, .04].

Expectancy-value beliefs

At Follow-Up 1, students in the treatment condition reported stronger deliberate practice beliefs (practice-specific expectancy) than controls (M_treatment = 75.33, M_control = 72.90), t(752) = 2.07, p = .04, d = .20, 95% CI [.01, .40]. This effect persisted to Follow-Up 2, one month later (M_treatment = 76.38, M_control = 73.37), t(752) = 2.32, p = .02, d = .25, 95% CI [.05, .45]. Treatment participants also reported higher frustration tolerance (practice-specific cost, a component of value) than controls at Follow-up 1 (M_treatment = 3.88, M_control = 3.54), t(750) = 5.93, p < .001, d = .68, 95% CI [.47, .88], and Follow-Up 2 (M_treatment = 3.77, M_control = 3.48), t(750) = 5.62, p < .001, d = .60, 95% CI [.40, .80].

Deliberate practice behavior

At Follow-Up 1, two to three days following the intervention, students in the treatment condition did more deliberate practice than controls (M_treatment = 944.80, M_control = 889.94), t(380) = 2.18, p = .03, d = .25, 95% CI [.01, .52]. This effect persisted to Follow-Up 2, one month later (M_treatment = 926.24, M_control = 861.50), t(380) = 2.68, p = .01, d = .29, 95% CI [.01, .57].

Academic achievement

Students in the treatment condition earned higher fourth quarter GPAs than controls (M_treatment = 0.09, M_control = −0.03), t(426) = 2.21, p = .03, d = .21, 95% CI [.02, .40]. This effect was driven by lower-achievers, Condition X Prior achievement interaction, b = −1.14, SE = 0.47, t(415) = −2.43, p = .02 (see Figure 4). A follow-up Johnson-Neyman analysis (Hayes, 2013; Johnson & Neyman, 1936) revealed that the effect of treatment was significant among lower-performing students (< 0.12 SD above average on prior achievement).

Figure 4.

In Study 4, the intervention increased fourth quarter grades for all sixth graders, and especially among lower-achievers. Shaded areas indicate regions of significance.

Because the intervention’s effect on end-of-quarter achievement was driven by lower-achievers, likely due to ceiling effects, we only expected changes in beliefs and behaviors to mediate the intervention’s effect on achievement in this lower-performing sub-group. Unfortunately, the current sample was underpowered to test for moderated mediation (Fritz & MacKinnon, 2007) (for further details and exploratory analyses, see the online supplement).

Study 5

In Study 4, an expanded deliberate practice intervention delivered to sixth graders close to the start of the fourth quarter improved practice-specific expectancies and values, deliberate practice behavior, and fourth quarter achievement. In Study 5, we aimed to replicate these findings and test boundary conditions. Would the intervention have the same effect among seventh graders in a different district? Whereas in Study 4 we assessed beliefs and behaviors one month later, in Study 5 we conducted a follow-up assessment four months later. Testing the same intervention in a different district, with an older population, using an extended timeline, we probed the generalizability and longevity of the intervention’s effects.

We introduced one final innovation in Study 5. Students were randomized to one of three conditions: full treatment, half treatment, or control. Full treatment participants received the same two modules as the treatment group in Study 4. Control participants received the same two modules as the control group in Study 4. The half treatment was a hybrid. Participants in the half treatment condition received the first treatment module (treatment condition, Part I) followed by the second control module (control condition, Part II). Thus, participants in the half treatment condition learned about deliberate practice, but did not receive the motivational content that targeted expectancies and values. We included this hybrid condition to parse the effects of the first treatment module from the second. Would teaching the tenets of deliberate practice suffice to change behavior (half treatment) or does deliberate practice need to be motivated (full treatment)?

Method

Participants

A total of 248 seventh grade students at a public school district in the U.S. were invited to participate. Of these students, approximately 94% (N = 232) took the intervention. The remaining 6% of students either opted out, experienced technological malfunctions (e.g. computer crashed), or were absent. Administrators in Study 5 made every effort to schedule make-ups, thus minimizing the number of non-participants. Participants and non-participants did not differ on baseline variables (e.g., gender, ethnicity, first quarter GPA). For details, see the online supplement.

Approximately 65.0% of participants were White, 13.8% Black, 10.3% Asian, and 10.3% Hispanic; 52.6% were female. Twelve percent of students qualified for free or reduced price meals.

Procedure

Overview

Procedures were similar to those in Study 4. In the fall, students were administered a battery of self-report scales and the Deliberate Practice Task (DPT) during three periods over six school days. Approximately eight weeks later, at the end of the second quarter, students completed the intervention during two periods spread across three school days. Since all classes in the district operated on a two-day cycle (A-day/B-day), students participated every other day. One week after the intervention (Follow-up 1), and then again four months later (Follow-up 2), students re-took the same battery of measures they completed pre-intervention. As in Study 4, students wore ear buds during all study sessions and privacy screens were attached to all computers.

For approximately 60% of students (B-day students at both middle schools, and A-day students at one middle school), the DPT was cancelled due to snow. Following the protocol described in Study 4 (refer there for details), DPT data collected during and after re-scheduled sessions were excluded from analyses. The percentage of students with missing self-report data ranged from 4–13% across the three data collections. Participants with missing data (e.g., DPT data, self-reported beliefs) did not differ from participants without missing data on any baseline variables (e.g., gender, ethnicity, first quarter GPA). For further details, see the online supplement.

Intervention procedures and content

Students were randomized to one of three conditions: full treatment, half treatment, or control. Full treatment participants received the same two modules that the treatment group received in Study 4. Control participants took the same two modules that the control group took in Study 4. The half treatment received a mix: the first treatment module (treatment condition, Part I, which taught the tenets of deliberate practice) followed by the second control module (control condition, Part II, which discussed interests and achievement). Part I and Part II of the treatment and control modules are described in greater detail in Study 4.

Measures

Self-report questionnaires

The self-report measures from Study 4 were re-administered in Study 5. Once again, students took practice-specific measures of expectancy (deliberate practice beliefs at Follow-up 1, α = .57; at Follow-up 2, α = .67), and cost (frustration tolerance at Pre-intervention, α = .50; at Follow-up 1, α = .64; at Follow-up 2, α = .57), as well as non-practice-specific measures of expectancy (growth mindset at Pre-intervention, α = .67; at Follow-up 1, α = .78; at Follow-up 2, α = .85; locus of control at Pre-intervention, α = .45; at Follow-up 1, α = .67; at Follow-up 2, α = .65), and cost (distress tolerance at Pre-intervention, α = .85; at Follow-up 1, α = .88; at Follow-up 2, α = .89). Due to time constraints, deliberate practice beliefs were not measured pre-intervention.

Deliberate practice task (DPT), time focused and total task time

Participants completed the same task described in Study 1.

Grade point average (GPA), standardized test scores, and basic demographics

Because the intervention was administered towards the end of the second quarter, we assessed the intervention’s effect on third quarter GPA controlling for GPA from the prior quarter (first quarter). GPA from the first and third quarters, standardized achievement test scores from the prior year, as well as gender and ethnicity were obtained from school records.

Interest

As in Study 4, students rated how interesting they found each of the two modules (r = .75, p < .001) and these ratings were averaged to create a single measure of student interest.

Results and Discussion

Across reported analyses, the effect of half treatment did not differ systematically from the effect of full treatment or control. Results for half treatment participants are reported following results for full treatment participants.

Preliminary analyses

No baseline differences emerged between groups on any measured variables (for details, see online supplement). Likewise, a one-way ANOVA revealed no between-condition differences in how interesting students found the modules, F(2,229) = 0.76, p = .47, η² = .01.

Expectancy-value beliefs

At Follow-up 1, full treatment participants reported higher deliberate practice beliefs (M_{full treatment} = 67.36, M_{half treatment} = 68.77, M_control = 62.55), t(208) = 2.19, p = .03, d = .36, 95% CI [.03, .69], and frustration tolerance (M_{full treatment} = 3.91, M_{half treatment} = 3.52, M_control = 3.51), t(396) = 2.05, p = .04, d = .32, 95% CI [.01, .64], compared to controls. Neither the effect on deliberate practice beliefs (M_{full treatment} = 75.73, M_{half treatment} = 76.95, M_control = 71.63), t(188) = 1.14, p = .26, d = .20, 95% CI [−.14, .53] nor the effect on frustration tolerance, (M_{full treatment} = 3.80, M_{half treatment} = 3.59, M_control = 3.64), t(396) = −0.11, p = .91, d = −.03, 95% CI [−.36, .31] persisted to Follow-up 2.

Half treatment participants reported higher deliberate practice beliefs, t(208) = 2.88, p = .004, d = .46, 95% CI [.14, 79], but not higher frustration tolerance, t(396) = −0.21, p = .83, d = −.02, 95% CI [−.34, .29] at Follow-up 1. At Follow-up 2, half treatment participants did not differ from controls on deliberate practice beliefs, t(188) = 1.48, p = .14, d = .25, 95% CI [−.09, .59] or frustration tolerance, t(396) = −0.64, p = .52, d = .11, 95% CI [−.22, .45].

Deliberate practice behavior

At Follow-Up 1, students in the full treatment condition trended towards doing more deliberate practice than controls, but this effect was not statistically significant (M_{full treatment} = 1312.40, M_{half treatment} = 1061.82, M_control = 1115.00), t(96) = 1.84, p = .07, d = .46, 95% CI [−.19, 1.13]. At Follow-Up 2, four months later, there was no effect of treatment (M_{full treatment} = 1232.10, M_{half treatment} = 1121.47, M_control = 1156.47), t(96) = 0.50 p = .61, d = .13, 95% CI [−.54, .81].

For students in the half treatment condition, there were no significant main effects at Follow-up 1, t(96) = −0.61, p = .55, d = −.13, 95% CI [−.73, .47] or Follow-up 2, t(96) = −0.45, p = .72, d = −.08, 95% CI [−.73, .56].

Academic achievement

There was no main effect of full treatment (M_{full treatment} = 0.08, M_{half treatment} = −0.02, M_control = −0.05), t(217) = 1.47, p = .14, d = .25, 95% CI [−.03, .52], nor was there a Full treatment X Prior achievement interaction, b = −0.15, SE = 0.09, t(217) = −1.63, p = .11. Nevertheless, because the interaction was in the same direction as the interaction in Study 4, we probed it to see if the same pattern of results emerged. It did. A follow-up Johnson-Neyman analysis (Hayes, 2013; Johnson & Neyman, 1936) revealed that the effect of full treatment was significant among lower-performing students (<−0.31 SD below average on prior achievement) (see Figure 5).

Figure 5.

In Study 5, the full treatment condition increased third quarter grades for lower-achieving seventh graders. Shaded areas indicate regions where full treatment differed from the control condition (half treatment did not differ from control at any values of the moderator).

For students in the half treatment condition, there was no significant main effect of intervention, t(231) = 0.27, p = .79, d = .06, 95% CI [−.21, .33]. Nor was the intervention effect moderated by prior achievement: Half treatment X Prior achievement interaction, b = −0.16, SE = 0.11, t(231) = −1.48, p = .14.

Because the intervention’s effect on end-of-quarter achievement was driven by lower-achievers, we only expected changes in beliefs and behaviors to mediate the intervention’s effect in this lower-performing sub-group. Yet, as in Study 4, the sample was underpowered to test for moderated mediation (Fritz & MacKinnon, 2007) (for exploratory analyses, see the online supplement).

Synthesis of Study Results

Because the size and significance of intervention effects on achievement varied across the four intervention studies—two of the four main effects, and three of the four Condition X Prior achievement interactions reached significance (the fourth interaction approached significance)—we conducted two meta-analyses using the “metafor” package in R. In the first meta-analytic model, we tested whether there was a significant mean difference on post-intervention achievement between the treatment and control groups across the four interventions. Effect sizes were calculated using Hedges’ g. A mean effect size (g₊) was calculated by weighting each study effect size by its inverse variance and averaging across the weighted estimates (Lipsey & Wilson, 2001). Given the small number of studies in the meta-analysis, we combined means using a fixed-effects model. We found a small but significant mean effect (g₊ = 0.20; p = .002, 95% CI [.08-.33]) and found little evidence of heterogeneity across studies (Q(3) = 1.02, p = .80). This suggests that, on average, the intervention improved achievement outcomes (see Figure 6a). We then ran a random-effects model as a sensitivity analysis using the DerSimonian-Laird estimator. The random-effects model showed little evidence of between-study variability (I² = 0%) and yielded identical mean effect size estimates and confidence limits to the hundreds decimal place.

Figure 6.

On average, across Studies 2–5, mean achievement scores were 0.20 standard deviations higher in the treatment conditions than control conditions at post-intervention (Panel a). Additionally, the weighted mean coefficient for the Treatment X Prior achievement interaction was significantly different than 0 (Panel b).

Next, we ran a meta-analytic model testing whether the mean interaction coefficient for the Condition X Prior achievement interaction was significant across the four studies. Regression coefficients for the interaction effects were weighted by their inverse variance and pooled using a fixed-effects model. The mean coefficient for the Condition X Prior achievement interaction was significant (b₊ = −.18; p < .001, 95% CI [−.27, −.10]) and we found little hetereogeneity across studies (Q(3) = 4.25, p = .24). This suggests that the magnitude of the intervention effect differed as a function of prior achievement levels (see Figure 6b). A random-effects model using the DerSimonian-Laird estimator yielded highly similar results (b₊ = −.20; p = .0004, 95% CI [−.31, −.09]).

General Discussion

In the current investigation, a psychologically “wise” deliberate practice intervention improved expectancy-value beliefs, deliberate practice, and academic achievement among non-experts. In Study 1, we validated a behavioral measure of deliberate practice and confirmed the importance of deliberate practice to non-expert achievement: Middle school students who did more deliberate practice earned higher grades. Next, across four longitudinal, randomized-controlled, field experiments, we intervened. In Study 2, a brief, wise intervention improved the rate at which lower-achieving fifth and sixth graders mastered math in Khan Academy. In Study 3, an adapted version of the same intervention delivered to undergraduates raised end-of-semester grades, particularly for lower-performing students. In Study 4, an expanded intervention improved sixth graders’ practice-specific expectancy-value beliefs, deliberate practice behavior, and academic achievement, with the strongest effects on achievement found among lower-performing students. A similar pattern of results emerged among seventh graders in Study 5. In sum, across four intervention studies we analyzed effects on three key outcomes: beliefs, deliberate practice, and achievement. The intervention changed beliefs and behaviors for one month (Study 4) but not four (Study 5), and improved achievement—particularly among lower-achievers—over one academic quarter.

Positive effects occurred regardless of whether the intervention was delivered to students in middle school (Studies 2, 4, 5) or college (Study 3). Likewise, the intervention affected a range of achievement-related outcomes: among lower-performing students, the intervention improved math performance in Khan Academy (Study 2), end-of-course grades (Study 3), end-of-semester GPA (Study 3), and end-of-quarter GPA (Study 4; the effect was in the same direction but not significant in Study 5). To synthesize effects on achievement, we conducted a meta-analytic estimate of the main effect and the interaction effect across all four studies. We found a small but significant main effect and a significant Condition X Prior achievement interaction.

The treatment condition proved beneficial above and beyond two control conditions, one that instructed students in standard study skills (Studies 2 and 3), and the other that discussed deep interests, generalized motivation, and high achievement (Studies 4 and 5). Across studies, teachers were blind to hypotheses, condition assignment, and intervention content. Thus, we believe we can rule out the possibility that gains in achievement were confounded by teacher expectancy.

Contributions of the Present Research

On a theoretical level, this research supports the heretofore untested hypothesis that deliberate practice submits to the same motivational principles that govern ordinary effortful behavior. Across a series of randomized-controlled experiments, wise interventions targeting expectancies and values motivated deliberate practice and improved achievement among non-expert students. Past research documents that expectancy and value are associated with persistence and performance (Eccles, 1983; Eccles & Wigfield, 2002; Eccles et al., 1984; Wigfield & Eccles, 2000); yet to our knowledge, the current investigation is the first to demonstrate that manipulating these variables can increase deliberate practice. Though our findings suggests that exceptional levels of grit and passion are not required to motivate deliberate practice, it appears some form of motivation is. In Study 5, students who were taught the tenets of deliberate practice without an accompanying motivational lesson (targeting self-relevant expectancies and values) did not improve their deliberate practice behavior or their achievement.

The Deliberate Practice Task (DPT) represents an important methodological advance. This novel task measures deliberate practice among novice students. Whereas standard self-report measures of deliberate practice (e.g., retrospective estimates, diaries) rely on participants to accurately report the quality of their practice—requiring metacognitive insight that is present among experts but not necessarily among non-experts—the DPT circumvents this need by assessing behavior objectively. Task measures have their own limitations (e.g., practice effects) (Duckworth, Yeager & Bryk, 2014). However, we believe the DPT provides a useful counterpart to self-report measures. In the present investigation the DPT featured math problems, but other content could easily be substituted to adapt the task to measure deliberate practice in other domains (e.g., science, literacy).

Finally, the present investigation makes a timely practical contribution. A brief, wise intervention improved the academic performance of students in middle school and college. Whereas the intervention raised achievement for all students in some studies (Studies 3 and 4), the intervention consistently improved the academic performance of lower-achievers (Studies 2, 3, 4 and 5).

Limitations and Future Directions

Limitations of the current investigation suggest several profitable directions for future research. First and foremost, our intervention did not turn non-experts into experts. World-class experts average several hours of deliberate practice per day, over the course of years (Ericsson, 2006). Contrast this with students in the current investigation who increased their deliberate practice for one month (Study 4) but not four (Study 5). As opposed to transforming novices into experts, what these studies provide is proof of concept: brief, motivational interventions can promote short-term increases in the amount of deliberate practice non-experts voluntarily perform. Future research is needed to determine precisely how long effects last, the mechanism(s) by which they attenuate, and whether booster sessions can halt this decline.

Future research is also needed to determine whether the intervention can raise achievement for all students. In the current investigation, the intervention preferentially helped lower-achievers (Studies 2, 3, 4 and 5). We believe this was due to ceiling effects. Measures of achievement could not capture improvement among students who were excelling prior to the intervention. Because almost all individuals in non-expert settings do less deliberate practice than they are capable of, future studies ought to assess achievement outcomes that do not have restriction on range. On such outcomes, we expect the intervention would more evenly benefit all students.

How can future interventions be strengthened? One possibility is to re-design the intervention to target the individual and the broader context. The present intervention taught students to execute deliberate practice absent teacher involvement—participation involved interacting with a computer. In the real world, even experts execute deliberate practice in conjunction with experienced others (e.g., coaches) in their surroundings. It will be interesting for future studies to measure the effects of more expansive interventions—those which address not just students but also teachers, the grading system, and the broader school culture.

A second possibility is to see whether combining the present intervention with other active ingredients has synergistic effects. Despite obvious similarities between the present intervention and growth mindset interventions—both target achievement-related outcomes and aim to recalibrate attitudes to challenge—the present intervention had no reliable effect on self-reported growth mindset. Given this non-overlap, it is possible that the present intervention, combined with a growth mindset intervention, would have an additive, even multiplicative, effect.

The various parts of the present, multi-faceted intervention still need to be systematically disentangled. Deliberate practice encompasses a suite of behaviors (e.g., setting specific goals, seeking out feedback, working on weaknesses). Instructing students on any of these behaviors may have driven the results. Feedback, for example, is robustly associated with learning and achievement (Bangert-Drowns et al., 1991; Hattie & Timperley, 2007). Likewise, self-testing, an activity consistent with deliberate practice, independently promotes learning and performance (Roediger & Karpicke, 2006). Against these specific possibilities, up to one month following the intervention, students were more likely to choose to challenge themselves while working (as measured by the DPT), a behavior that does not directly follow from learning to self-test or seek feedback. Nevertheless, more investigations like Study 5 that parse the intervention to identify active ingredients are warranted.

For a number of reasons, we also recommend replications. One important reason to conduct replications is because the present intervention was, to some degree, customized. Focus groups were drawn from the schools in which the interventions were run. It would be interesting to know whether an intervention that has not been customized to the participating student body leads to similar results. In addition, the present studies suffered from a number of irregularities (e.g., missing data, snow day cancellations). Replications, particularly in samples that are more diverse, would provide further evidence of generalizability.

Finally, we recommend replications in larger samples, which would permit tests of moderated mediation. Since the intervention only consistently improved achievement for lower-performing students, one would expect to find the intervention’s effect on achievement mediated by a change in beliefs and behaviors in this subsample. Moderated mediation analyses, which the current studies were underpowered to perform, would permit a more conclusive determination of whether the beliefs and behaviors that changed post-intervention explain the achievement gains of lower-performing students. A current frontier in the study of wise interventions is to identify the precise pathways through which they generate enduring effects (Yeager & Walton, 2011; Walton, 2014). Thus, mechanisms that account for the current intervention, like those of other wise interventions, require continued exploration.

Conclusion

Deliberate practice was first investigated as a predictor of excellence among experts (Ericsson et al., 1993). To a large degree, the domain of expertise is where the study of deliberate practice has remained. We hope our findings goad researchers and educators alike to wonder, why? Why not teach this method of continuous improvement to, well, everyone? In the present investigation, a brief intervention anchored in psychological theory changed students’ beliefs, deliberate practice, and academic achievement for up to one academic quarter. Future research is needed to determine whether these effects can be strengthened, and if so, to what extent. Can deliberate practice interventions transform novices into experts, or merely make them expert-like? Answers to these questions will reveal the full extent to which deliberate practice, and excellence itself, can be encouraged among non-experts.