How Can We Inspire Nations of Learners? Investigating Growth Mindset and Challenge-Seeking in Two Countries

Abstract

Here we evaluate the potential for growth mindset interventions (which teach students that intellectual abilities can be developed) to inspire adolescents to be “learners”—that is, to seek out challenging learning experiences. In a previous analysis, the U.S. National Study of Learning Mindsets (NSLM) showed that a growth mindset could improve the grades of lower-achieving adolescents, and, in an exploratory analysis, increase enrollment in advanced math courses across achievement levels. Yet the importance of being a “learner” in today’s global economy requires clarification and replication of potential challenge-seeking effects, as well as an investigation of the school affordances that make intervention effects on challenge-seeking possible. To this end, the present paper presents new analyses of the U.S. NSLM (N = 14,472) to (a) validate a standardized, behavioral measure of challenge-seeking (the “make-a-math worksheet” task), and (b) show that the growth mindset treatment increased challenge-seeking on this task. Second, a new experiment conducted with nearly all schools in two counties in Norway, the U-say experiment (N = 6,541), replicated the effects of the growth mindset intervention on the behavioral challenge-seeking task and on increased advanced math course-enrollment rates. Treated students took (and subsequently passed) advanced math at a higher rate. Critically, the U-say experiment provided the first direct evidence that a structural factor—school policies governing when and how students opt in to advanced math—can afford students the possibility of profiting from a growth mindset intervention or not. These results highlight the importance of motivational research that goes beyond grades or performance alone and focuses on challenge-seeking. The findings also call attention to the affordances of school contexts that interact with student motivation to promote better achievement and economic trajectories.

To thrive in the new labor market, we will need a citizenry with a desire for challenge and an ability to cope with difficulty. In the past, it may have seemed sufficient for people to be “knowers”—to know facts and specific skills and then apply them going forward. However, as technology makes many jobs obsolete (Acemoglu & Autor, 2011; Autor, 2014; Kraft & Grace, 2016), the jobs that technology creates require a thirst for challenge and learning (Deming, 2015). This is because routine tasks and well-defined problems can be taken care of with automated solutions using ever-more-sophisticated algorithms (Lu, 2015). Thus, it is critical that people also become “learners”—that they habitually seek out hard-to-acquire expertise that can help them succeed in the future (National Research Council, 2012).

This issue comes to the fore in high school, and particularly in math classes, which represent a fork in the road for many young people. Advanced math skills serve as a foundation for higher-level science, technology, engineering and math (STEM) courses and STEM professions later on (National Research Council, 2010, 2012). Even among those who do not go into STEM professions, math literacy can increase logical reasoning skills that can be applied broadly (National Research Council, 2010). Because advanced math exposure in high school opens the gateway to valued careers that in turn are associated with better health, advanced math is a strong early indicator of not only wealth but also well-being and longevity (Carroll & Muller, 2018). Yet students can choose to limit their exposure to challenging math content in high school (Carroll et al., 2017; Schiller et al., 2010). One way they may do so is by opting out of math classes that might take them out of their comfort zone.

We have seen the phenomenon of avoiding math challenges first-hand in our research. On a survey with a nationally-representative sample 9^th grade students in the U.S (Yeager, 2019), we included a survey question that assessed a desire to embrace challenging math. We describe the experimental results later, but for now the control group’s choices (N = 7,215) are informative. We offered adolescents a hypothetical choice between two kinds of extra credit math assignments—one with math problems that were easily done without much thinking and the other with problems that were very challenging but would promote learning. Fully 63% of 9^th grade students in the U.S. chose the easy math assignment that would teach them nothing new, meaning that only 37% chose the hard one that they could learn from. Data we present below from Norway also show considerable under-utilization of rigorous opportunities to learn. Thus, avoiding math challenges is not only a U.S. phenomenon.

The finding that nearly two-thirds of U.S. 9^th graders avoided a challenging math assignment (even when nothing was at stake) is noteworthy because the purpose of education is, of course, to expand skills and knowledge. Hence, even if schools and teachers were offering opportunities for students to deepen their knowledge and push beyond their current skill levels, many students might nevertheless fail to embrace those learning opportunities. As we have noted, this will make them less well-prepared for the realities of the current and future global economy than they otherwise could be.

A major goal for policy and for interdisciplinary behavioral science, then, is to motivate adolescents to take on the challenges that are being presented to them in high school. In other words, how can we begin to inspire “nations of learners?”

In the present research we evaluated the potential for growth mindset interventions to inspire challenge-seeking in population-generalizable samples of high school students. The growth mindset is the idea that intellectual abilities are not fixed, and that it is possible, through learning, to develop stronger abilities—that is, a stronger brain (J. M. Aronson et al., 2002; Blackwell et al., 2007; Paunesku et al., 2015; Yeager et al., 2016). Growth mindset interventions invite students to learn scientific information about the potential to develop one’s intellectual ability and the brain’s potential to form new or stronger neural connections when it learns (J. M. Aronson et al., 2002; Blackwell et al., 2007; Paunesku et al., 2015; Yeager et al., 2016). Students then reflect on what this means for their learning, including how their neural connections could develop and grow stronger when they try hard on challenging work, change their learning strategies, or ask for help from others (Yeager & Dweck, 2012).

The effect of these mindset messages is to change the meaning of challenges, so they are seen as opportunities for students to grow their intellectual abilities (Hong et al., 1999; Molden & Dweck, 2006), not as threats to their sense of their abilities. When students believed that their abilities could be developed, challenging assignments had a different, more positive, meaning, and setbacks were less likely to result in the attribution that one lacks raw intelligence (Blackwell et al., 2007; Hong et al., 1999). And students have felt more free to adopt the goal of learning (even when faced with the possibility of failure) rather than adopting the goal of avoiding failure by selecting tasks that are easy for them (Blackwell et al., 2007; Robins & Pals, 2002).

In laboratory and field research, students’ mindsets have been associated with outcomes at multiple levels of analysis (Dweck & Yeager, 2019). A fixed mindset has been related to neural systems implicated in mistake-processing, such that those with a fixed mindset engaged in less processing of error feedback when they had an opportunity to revise a mistake (Moser et al., 2011). At a neuroendocrine level of analysis, students with a fixed mindset showed more of a “threat” response to poor academic performance, in the form of higher cortisol levels, relative to students with more of a growth mindset (Lee et al., 2019). Further, students’ mindsets cause different metacognitive tendencies. For instance, those with more of a fixed mindset tend to compare themselves to those below them (so they can feel better than poor performers), while those with a growth mindset tend to compare themselves to people who did better than them (so they can learn more effective strategies) (Nussbaum & Dweck, 2008). Finally, students who learn about the growth mindset message have shown increases in daily motivated behavior, such as trying harder on a math quiz (Bettinger et al., 2018), revising one’s work, or staying after class for extra help (Blackwell et al., 2007).

It has recently become possible to deliver growth mindset interventions using relatively short, self-administered online modules lasting under an hour. For instance, the National Study of Learning Mindsets (NSLM) (Yeager, 2019) evaluated a growth mindset intervention in a nationally-representative sample of U.S. public schools (Yeager et al., 2019). The focus of the intervention was on improving grades and indeed it improved lower-achieving 9^th grade students’ school performance at the end of the school year. An exploratory analysis also showed that the intervention increased the rate at which students overall were enrolled in advanced math the next year. Although promising, this latter finding requires replication. And intervention effects on grades alone did not mean there were effects on challenge-seeking. Indeed, students may choose easier courses to help ensure higher grades.

Given the tremendous repercussions of being a “learner” for individual and societal economic success, a high priority for research is to clarify, verify, and extend our understanding of the role of growth mindset in challenge-seeking. We should ensure not only that the findings for advanced course taking from the NSLM are replicable and generalizable to other educational systems, but also that we understand the conditions under which the effects appear. Therefore, in the present research we answered three research questions, outlined next. We did so by conducting new analyses of the NSLM, and by conducting a new study, the U-say experiment, which is parallel to the NSLM and was conducted in Norway.

Research Questions

We first asked whether an online intervention that in principle could be scaled to an entire nation’s schools could inspire a willingness to be a “learner,” as assessed by a standardized task. The present study represents the most comprehensive test of this question to date. Past studies that have specifically assessed growth mindset and challenge-seeking have mostly used correlational methods (e.g. Blackwell et al., 2007; Dweck & Leggett, 1988; Robins & Pals, 2002). And the one study to compare a growth mindset treatment to a neutral control and show effects on challenge-seeking behavior used a much smaller sample of convenience (Bettinger et al., 2018). However researchers can only safely generalize the results of an experiment to a population (such as a nation or a region) when every person in the population had a known, non-zero probability of inclusion (Allcott, 2015; Kish, 2004). In the current research, we use two truly generalizable samples to go beyond past studies of growth mindset and challenge-seeking.

To answer our first question it was necessary to validate a task that could assess a student’s desire to be a “learner,” and we did this by analyzing data from the NSLM. We developed a behavioral marker of challenge-seeking tendencies with respect to high school math that could be administered efficiently in a national survey. We call it the “make-a-math-worksheet” task. Much like the delay of gratification task (aka the “marshmallow test”; Mischel, Ebbesen, & Raskoff Zeiss, 1972), behavior during our task serves as a marker for a broader construct (challenge-seeking). Also like the delay of gratification task, the make-a-math-worksheet task shows concurrent and predictive validity, as we will demonstrate.

Our second question, answered in the Norway study, was whether growth mindset interventions reliably affect the consequential decision to take advanced, theoretical math (versus easier, applied math) in the months following the intervention. Students in advanced math are more commonly asked to be “learners”—to apply deeper analysis, prove or justify their work, work on problems with multiple solutions, and generalize skills to new problems—while students in easier classes tend to focus on learning routine solutions and applying them (Carroll & Muller, 2018; Ferrare, 2013). We tested whether students who received the growth mindset would be more likely to sign up for or stay in challenging math courses. Although as noted an exploratory analysis of data from the NSLM showed effects on advanced course-taking, the more transparent nature of course decision-making processes and of course content in Norway makes it easier to be sure that challenge-seeking motivation could translate into students’ enrollment decisions. More specifically, the Norwegian context offers a clear choice between applied and theoretical math and therefore provides an important context for testing challenge-seeking hypotheses. (Because students could take classes they were unprepared for, we also looked at whether students eventually passed the more advanced math courses. This made no difference because so few students fail).

Third, the analysis of advanced math course-taking in the Norway study allowed us to answer a critical theoretical question about mindset interventions: how effects depend on the learning opportunities afforded by a school context (see Walton & Yeager, in press). Psychological interventions do not work in isolation but alter students’ beliefs and motivation within a given set of structural affordances (Cohen & Sherman, 2014; Walton & Wilson, 2018; Yeager & Walton, 2011). Mindset × Context Theory is a framework which makes specific predictions about the intersection of mindset interventions and affordances. This framework comes from an integration of theories of psychological interventions (Cohen & Sherman, 2014; Walton & Wilson, 2018; Yeager & Walton, 2011) with theories in the sociology of education (Carroll & Muller, 2018; Crosnoe & Muller, 2014).

Consider that not all students will be able to take advanced math, no matter how inspired to learn they become. Sometimes students are too far behind, but often it is structural factors that stand in their way. Sociological models of curricular differentiation (Carroll & Muller, 2018) point to structural “gateways” (such as when course selection takes place in relation to the intervention) or “gatekeepers” (teachers or counselors who decide who is eligible to take particular courses). Mindset × Context Theory predicts that a treatment that increased the motivation to be a “learner,” but did so in a context that made it hard to act on that desire, should be less likely to move students into challenging math pathways.

The Norwegian system offers an unprecedented opportunity to test this Mindset × Context interaction. School policies allow students to make the choice between theoretical or applied math either prior to entering high school (i.e. before the intervention) or several months into their first year of high school (i.e. after the intervention). We expected stronger growth mindset effects on advanced math enrollment in schools where the gateway was “open” (i.e. where students could freely choose their math track after the intervention) and weak or null effects when the gateway was “closed” (i.e. where students had already chosen their advanced math class, and could only be prevented from dropping down a level).

Anticipated Effect Sizes

What kinds of effects on advanced math course-taking should be expected? Adolescent behavior-change interventions in general tend to have null effects, even when they are costly and time intensive (Yeager et al., 2018), and so at some level any effect on a consequential outcome would be noteworthy. In terms of sizes of effects for the interventions that do produce benefits, noted psychologist Daniel Kahneman said “What you can hope for is what is called practically significant improvement, which is usually a few percent. If you get a few percent at relatively low cost, that’s a success” (Dubner, 2017). This statement is justified because some of the most successful “nudge” interventions aimed at changing future or ongoing behaviors (rather than immediate, one-time decisions), typically show effects in the range noted by Kahneman (Benartzi et al., 2017). A descriptive norm manipulation aimed at reducing energy use (the “Opower” experiments), led to a 0.5% to 2% reduction in kWh (Allcott, 2015). And implementation intentions interventions (which invite people to form concrete plans for how they will overcome later barriers to self-regulation) increased vaccination rates by 1.5% to 4.2% (Milkman et al., 2011). It would be informative if growth mindset interventions had an effect on the consequential and multiply determined outcome of advanced math course-taking in a similar range, and if the mindset effects were even larger when structural affordances opened the gateway to advanced course-taking.

Method

Data

U.S. Study.

The National Study of Learning Mindsets (NSLM) was conducted with first year high school students in a representative sample of U.S. public high schools. Detail on the methods for data collection are reported in publicly-available technical documentation (Yeager, 2019), in a description of the sampling plan (Tipton et al., in press) and in a previous report of different analyses of the NSLM data (Yeager et al., 2019), so we provide a more limited summary here. A third-party firm, ICF International, recruited all schools and collected all survey data; of 139 randomly-sampled high schools, 76, or 55%, participated. These schools were highly representative of the population (Gopalan & Tipton, 2018; Yeager et al., 2019). Students in those schools were invited to complete two online survey sessions (“Time 1” and “Time 2”); of those who started Time 1, 89% provided outcome data at Time 2, yielding a maximum analytic sample for the dependent measure of the worksheet task (described below) of N=14,472. Intervention and survey data were collected between August, 2015 and March, 2016.

Norway Study.

The U-Say experiment was conducted with public high schools in the Rogaland and Akershus counties of Norway (95 percent of all students attend public high schools in Norway). All schools in these two counties were invited to participate; 49 out of 50 academic-track high schools accepted the invitation. In Rogaland and Akershus the high school completion rates are 75 and 79 percent respectively; for Norway overall it is 73 percent. These counties are similar to the U.S. overall, where graduation is just over 80% (McFarland et al., 2016). The national test scores in Norway are also very similar to those in the U.S. (OECD, 2016). Thus the schools in this replication are similar in many ways to the U.S. context. Even with the similarities across contexts, an intensive R&D process was carried out to customize the intervention for the Norwegian population, as described elsewhere (Bettinger et al., 2018). The Norway experiment was conducted in the fall of 2017.

Consent from students was obtained from 90 percent of invited students. A total of N = 6,541 students aged 15 to 17 completed the Time 1 survey and were in the intent-to-treat sample.¹ Half were female. A total of 5,247 students completed the Time 2 survey and provided data on survey-measured outcomes. (Data were collected from a sample of vocational-school students as well but they could not be included in these analyses because vocational schools do not offer advanced math classes.)

Procedure

Student data collection.

Data collection and intervention delivery occurred via a website, which allowed all parties to be blind to treatment condition assignment. The two-session randomized experiment occurred during regular school hours. Each session—from here forward, “Time 1” and “Time 2”—lasted approximately 25 minutes and usually occurred one to four weeks apart. Time 1 involved brief baseline survey measures, followed by the first section of the growth mindset or control materials, followed by demographic measures. Time 2 involved the second section of the growth mindset or control materials, followed by the outcome measures used here.

Growth mindset intervention.

R&D.

The growth mindset online intervention for the NSLM was created through a two-year, iterative, design and prototyping process whose goal was to revise prior materials and create materials that would be effective across student groups in 9^th grade. Much of this R&D process is described in Yeager et al. (2016), and it involved a series of improvements to the content, the visual layout, the specific examples used in the intervention, the activities students engaged in, and so on. Piloting and R&D involved over 16,000 participants (see the online supplement). Next, an additional, intensive R&D process changed the intervention further for the Norway context; this is described by Bettinger et al. (2018).

Intervention content.

Additional detail on the growth mindset intervention can be found in previous papers published on it (Bettinger et al., 2018; Yeager et al., 2016, 2019). Here we provide a summary.

First, the growth mindset intervention presented evidence and arguments for the idea that doing challenging work can make one’s abilities stronger over time. After explaining how neurons work, the intervention informed students that the connections between neurons can be weak or strong. When students work hard to learn something new—like a new type of math problem—the connections in their brain can become more efficient (i.e. stronger). Next, the intervention provided published evidence that high school (adolescence) is a particular time when the brain can learn and grow—perhaps more than many other times in life. Finally, the intervention explained how building a stronger brain in high school can be helpful to people no matter what they plan to do in life. The reason for this is that in fashioning a growth mindset intervention for students from different racial, ethnic, and social class backgrounds, and at all levels of motivation and engagement, it was critical to help students reflect on the idea that they can grow intellectually and why they would want to. Past growth mindset interventions have assumed that all students wanted to develop their intellectual abilities and strengthen their brains. But this may not always be the case. In particular, a desire for sheer intellectual growth for its own sake may not be as strong for all gender, racial, and social-class groups (Diekman et al., 2010; Fryberg et al., 2013). Thus the present growth mindset intervention invited students to reflect on how a stronger brain could help them reach important goals in their lives (Yeager et al., 2014). The intervention also included stories from older high school students and from prominent individuals who described how they used their “stronger brains” to achieve important goals.

The intervention involved elements for creating internalization that are now common among social-psychological interventions (Walton & Wilson, 2018; Yeager & Walton, 2011). The intervention conveyed a sticky metaphor—the notion that the brain is like a muscle that grows in response to challenging experiences. It involved source credibility, such as quotes from psychological scientists and notable public figures who endorsed the notion that the brain develops when it learns and explained how a stronger brain could help them achieve their goals. The materials leveraged descriptive social norms by including quotations from past participants who explain the intervention messages (Cialdini & Goldstein, 2004). Finally, the intervention involved “saying-is-believing” or self-persuasion exercises (E. Aronson, 1999; J. M. Aronson et al., 2002) that, for example, invited participants to advise a future struggling 9th grade student in terms of the principles set forth in the intervention (e.g., when school is hard, it means you’re learning and gaining skills that will help you make a difference later). In addition to providing advice for peers, students completed a self-persuasion exercise in which they explained how they will use their stronger brains to achieve meaningful goals.

Control materials.

The control activity was designed to parallel the growth mindset activity. It, too, was framed as providing helpful information about the transition to high school. The control activity involved the same type of graphic art (e.g., images of the brain and animations), as well as compelling stories (e.g., about Phineas Gage). It taught basic information about the brain, which might have been useful to students taking 9th grade biology. It also provided stories from upperclassmen, reporting their opinions about the content. The stories and quotes from noted individuals in Time 2 were matched in source but differed in content across conditions. For instance, in the U.S. context, in the control activity former First Lady Michelle Obama talked about the White House’s BRAIN initiative, an investment in neuroscience, while in the growth mindset condition students read a speech given by the First Lady about how hard work in school can make you smarter. Finally, as in the growth mindset condition, there were opportunities for interactivity. Students were asked open-ended questions and they provided their reactions. Overall, the control condition was strong because it (a) controls for expectancy effects (it too conveys that learning is positive and important), (b) was able to maintain the double-blind design due to parallel content, (c) provided engaging scientific information that may have sparked an interest in learning about science in general or the brain in particular; (d) involved public-figure and upper-year student endorsements of learning goals; and (e) was autonomy supportive, in that it allowed students to write their own reactions throughout.

Measures

Challenge-seeking behavioral task.

Participants completed the make-a-math-worksheet task, a behavioral assessment (see Yeager et al., 2016). At a conceptual level, the task is designed to measure a willingness to opt in to more intellectually challenging experiences—ones that might lead to deeper knowledge and skill, even if it comes at the cost of slightly lower performance or the potentially unpleasant experience of feeling lost or confused. The task has been used in past research but not fully validated (this is done below).

In the U.S., at the end of the Time 2 session, students were asked which math class they were currently taking (Pre-Algebra or earlier; Regular Algebra 1; Advanced, Honors, or Pre-AP Algebra 1; Geometry; Above Geometry), and were then directed to view four (three in Norway) “chapters” of problems, each on a different topic within their course and all matched to their course level. In the Norway context, math problems were selected based on knowledge of students’ math curricula. Students then chose the problems that they wanted to solve from each of the chapters.

Each “chapter,” presented on its own page, included six problems to choose from, and each problem was described as either “Not very challenging, and you probably won’t learn very much” “Somewhat challenging, and you might learn a medium amount” or “Very challenging, but you might learn a lot” (two per type). For each chapter, students were instructed to select at least 2 and up to 6 problems on each page, and problems were presented in a random order for each student for each page (see Yeager et al., 2016 for screenshots). After making their choices, participants were told that unfortunately there would not be time to complete the problems, but they were thanked for their time and were told that their preferences were informative. In the Norway Study, students actually completed two randomly selected questions from their worksheets; results were the same.

The total number of “Very challenging” (i.e., hard) problems chosen across the 4 (3) pages was calculated for each student (Range: 0–8 in the U.S., 0–6 in Norway) as was the total number of “Not very challenging” (i.e., easy) problems (Range: 0–8 in the U.S., 0–6 in Norway). The pre-registered measure (osf.io/64srk/) was the difference between the number of hard problems and the number of easy problems selected (Range: −8 to +8 in the U.S., −6 to +6 in Norway). Higher values corresponded to greater challenge-seeking.

Hypothetical challenge-seeking.

For initial validation of the make-a-math-worksheet task, we compared students’ responses to a hypothetical choice of a math problem (summarized in the introduction to this paper), published previously in Yeager et al. (2016). This hypothetical measure is informative because it is a direct test of the psychology we targeted. Participants expressed their preference for one of two types of math homework, an easy one where they would not learn anything new or a challenging one where they could learn something new (0= The easy math assignment where I would get most problems right, 1= The hard math assignment where I would possibly learn something new).

Mindset manipulation check.

Three items administered at Time 1 and Time 2 constituted a baseline and immediate-post-test manipulation check, to test whether the intervention successfully reduced the belief that intelligence cannot change (“You have a certain amount of intelligence, and you really can’t do much to change it,” “Your intelligence is something about you that you can’t change very much,” and “Being a ‘math person’ or not is something that you really can’t change. Some people are good at math and other people aren’t.” (Response options: 1=Strongly disagree … 6=Strongly agree). We included the math-specific fixed mindset item because it matched the domain of the challenge-seeking task. In large surveys where it is only possible to administer a few items, researchers usually select items framed in terms of the fixed mindset (rather than the growth mindset) because these are thought to be less susceptible to socially-desirable responding (Dweck, 1999). At each time point, responses were averaged into a single scale with higher values corresponding to more fixed mindsets (α = 0.73 at Time 1 and α = 0.78 at Time 2).

Baseline measures.

Before random assignment, students completed various measures that are useful for evaluating the effectiveness of random assignment (i.e., balance tests). Items were selected because they were expected to be associated with motivation and challenge-seeking, and therefore if random assignment failed in terms of these it could challenge the validity of the experimental comparison. These were: self-reported prior achievement (see Kuncel, Crede, & Thomas, 2005, for validity), expectancies for success in school, interest in math, gender, race or ethnicity (in the U.S. study), and maternal education.

For a sub-set of schools in the U.S. (up to 66, depending on the measure) and for all of the schools in Norway, we were also able to obtain data on students’ prior achievement. In the U.S. study a dichotomous variable indicated whether a given student was lower-achieving (below-median) before the study, following the analysis plan for administrative data (osf.io/afmb6/). The reason for this is that prior research has found growth mindset effects on GPA only for lower-achieving students, but we pre-registered the expectation that students at all achievement levels would increase their challenge-seeking motivation (osf.io/64srk/).

Advanced math course enrollment.

U.S. Study.

In order to further validate the make-a-math-worksheet task in the U.S. study, we analyzed students’ highest 10^th grade math courses the school year after the intervention (e.g. Algebra I, Geometry, Algebra II/Trigonometry), obtained from 41 schools. Taking Algebra II/Trigonometry in 10^th grade, rather than Geometry or a lower-level math class, is a threshold for staying on track for finishing a rigorous portfolio of classes by the end of high school (Ingels et al., 2015). As noted earlier, another manuscript reports the effects of the growth mindset intervention on taking Algebra II in 10^th grade (Yeager et al., 2019).

Norway Study.

After the school year was over and students had finished their year in advanced math, we obtained data on math course enrollment from administrative records. Government records reported whether students had taken advanced, theoretical math (which involves greater challenge and can lead to deeper learning and to careers and majors in math or science) or non-advanced, applied math (which typically involves applying mathematical routines and principles to real-world problems, and does not lead to math or science careers).

The critical between-school difference is that, in some schools, students made their choices for high school math about two months after having started high school. This was before the intervention and while the students were all in a generic math class (post-treatment choice schools, about one fourth of schools). These are the schools that present the greatest opportunity for a growth mindset intervention to increase rates of taking challenging math. In other schools, the choice of math course was made prior to the start of school (pre-treatment choice schools, about three fourths of schools). As in U.S. schools, it was theoretically possible for a treatment effect to still appear in the pre-treatment choice schools (because students could drop down a level), but it was unlikely. In terms of effect sizes, we expected small or null effects on actual course-taking in pre-treatment choice schools, even if students’ desire to embrace challenges (as measured by the behavioral task) was lifted.

To replicate the results from Yeager et al. (2019), our primary measure focused on students’ enrollment in the more advanced math course over the entire first year of high school. But as noted students would not necessarily be better off if the intervention caused them to take a class that they ultimately failed, and so we explored the results when instead examining whether students took and passed the advanced math class (1) versus all others (0).

Analysis Plan

Statistical tests come from cluster-robust fixed effects models that controlled for demographics and prior achievement, with cluster defined as the school. Robustness tests are reported in the online supplement. Descriptive statistics are estimated from the raw data; significance tests come from regression models. We report standardized effect sizes (d) by dividing the difference between the growth mindset and control conditions estimated in the regression model by the raw standard deviation of the control group. For the U.S. study, hypotheses and analysis methods were pre-registered (osf.io/64srk/). For the Norway study, we followed the same analysis methods as the U.S. study to constrain researcher degrees of freedom, but we adapted models to accommodate differences between the populations (e.g. it is not possible to control for race / ethnicity in the same way in Norway as in the U.S.).

Results

Preliminary Analysis: Validating the Make-a-Math-Worksheet Task

We first used data from the NSLM to validate the make-a-math worksheet task. For ease of presentation, we show results for students who fell into two groups: (1) those who selected more easy than hard math problems, and (2) those who selected more hard than easy math problems. We then show that same results hold in regression analyses that analyze the continuous measure of behavior on the worksheet task. Validity analyses for alternate operationalizations of the challenge-seeking measure are included in the online supplement.

Comparing the worksheets to the hypothetical scenario, we found that among students creating worksheets with more hard than easy problems, 57% said they would have chosen the harder extra credit math assignment on the hypothetical scenario, compared to 33% among those who created worksheets with more easy than hard problems, a significant difference, χ²(1, N = 11778) = 677.32, p < .001. Thus, the two challenge-seeking measures converged. Additional concurrent validity evidence comes from a conceptual replication of past research showing an association between mindsets and challenge-seeking. In this nationally-representative sample, a fixed mindset measured at Time 2 predicted less challenge-seeking behavior on the make-a-math-worksheet task, r(14084) = −.14, p <.001.

Another validity analysis of the make-a-math-worksheet task examined whether responses predicted advanced math course taking in 10^th grade, assessed via administrative records, accounting for 9^th grade math courses. Even among students who were not taking advanced math in 9^th grade, according to administrative records, those who created worksheets with more hard than easy problems had a 15% chance of being in Algebra II or above in 10^th grade, relative to 9% among those who created worksheets with more easy than hard problems, a significant difference,χ²(1, N = 3381) = 23.96, p < .001. Among students who took advanced math in 9^th grade, these numbers were 92% and 89%, respectively, also a significant difference, χ²(1, N = 1554) = 4.13, p = .042. This is good evidence for the validity of the make-a-math worksheet task because the differences in 10^th grade course taking appeared despite students’ 9^th grade math course levels.

These validity analyses were furthermore confirmed when predicting the two criteria with the full, pre-registered, continuous measure of behavior on the worksheet task (total hard problems chosen minus total easy problems; Range −8 to +8) in cluster-robust fixed effects linear probability regression models controlling for 8^th grade test scores and grades. Worksheet behavior significantly predicted both hypothetical challenge-seeking, linear probability model b = .035, SE = .001, t = 30.76, N = 13994, p < .001, and 10^th grade Algebra II or above, b = .008, SE = .002 t = 5.29, N = 5852, p < .001 (controlling for 9^th grade math course level and prior math grades and test scores).

Interestingly, the association of task choices with course-taking choices also appeared at the school level. In analyses presented in the online supplement, schools where students created more challenging worksheets also had much higher test-taking rates for AP Calculus (a challenging math class) according to administrative data, even controlling for school test scores and racial composition (see the online supplement). Altogether, these analyses show that the make-a-math-worksheet task—which is brief and readily scalable—can assess students’ readiness to try harder problems that might teach them something new—that is, their willingness to be “learners.”

Research Question 1: Effects of the Mindset Intervention on Challenge-Seeking

U.S. Study.

The NSLM provided a reasonable opportunity to test our study’s challenge-seeking hypotheses, because the growth mindset intervention significantly decreased students’ reports of fixed mindsets (Control M = 2.916, SD = 1.167; Growth mindset M = 2.515, SD = 1.175), t(14459)=24.93, p<.001, d=.332, as expected. And an analysis of hypothetical behavior showed that, in the control condition, only 37% of students said they would choose the difficult math assignment, while 49% in the growth mindset condition did so, a significant difference, χ²(1)=196.95, p<.001.

More importantly, an analysis of challenge-seeking behavior found that the growth mindset intervention increased the number of hard problems that students chose (Control Hard M=2.76, SD=2.41; Growth mindset Hard M=3.18, SD=2.54), unstandardized b = 0.44, t(14115) = 9.35, p < .001, and significantly reduced the number of easy problems chosen (Control Easy M=3.86, SD=2.53; Growth mindset Easy M=3.43, SD=2.52), b = −0.43, t(14115) = −10.51, p < .001, resulting in an intervention effect on the difference between the number of hard versus easy problems (Control difference score M=−1.09, SD=3.67, Growth mindset difference score M=−0.24, SD =3.83), unstandardized b = 0.87, t(14115) = 12.14, p < .001, d=.24. The intervention effect on behavior on the make-a-math-worksheet task did not vary significantly across student groups: there were no significant interactions with gender, race/ethnicity, parental education, 9^th grade math course level, or student status as a previously-lower-achieving student, ps > .05.

The finding that students at every level of achievement can increase their challenge-seeking by about a quarter of a standard deviation is important for clarifying past published analyses of the current dataset, which showed effects on grades only for lower-achieving students (Yeager et al., 2019). It would be a misinterpretation to say that growth mindset is only effective for such students, because challenge-seeking effects appeared overall. Moreover, these behavioral results clarify the role that challenge-seeking motivation may have played in previously-reported effects on math course-taking in the NSLM.

Norway study.

As a preliminary matter, the growth mindset intervention significantly decreased students’ reports of fixed mindsets (Control M = 2.52, SD = 1.00; Growth mindset M = 2.16, SD = 0.97), b = 0.362, t(5246)=13.30, p<.001, d=.32, and increased hypothetical challenge-seeking behavior: in the control condition, 59% of students said they would choose the difficult math assignment, while 68% in the growth mindset condition did so, a significant difference, χ²(1, N = 5241)=41.50, p<.001. Thus the intervention was persuasive, just as it was in the U.S.; this is a testament to the efficacy of the design process used to adapt it.

Next, replicating the US study, the growth mindset intervention significantly increased the number of hard problems that students chose (Control Hard M=2.734, SD=1.779; Growth mindset Hard M=3.071, SD=1.879), b = 0.330, t(5322) = 7.52, p<.001, and significantly reduced the number of easy problems they chose (Control Easy M=2.038, SD=1.974; Growth mindset Easy M=1.713, SD=1.884), b = −0.322, t(5322) = 5.72, p<.001, resulting in an intervention effect on the difference between the number of hard versus easy problems control difference score M=0.6953, SD=3.210, Growth mindset difference score M=1.358, SD =3.097), b = 0.652, t(5247) = 7.40 p<.001, d=.18.

As in the US study, this treatment effect was not moderated by student gender, prior math grades, or school type (pre- vs. post-treatment choice) (interaction ps>.5). This non-significant moderation is important because it shows, again, that a growth mindset intervention can increase a desire for challenge regardless of students’ prior achievement.

Research Question 2: Growth Mindset Effects on Advanced Math Course-Taking

Before assessing the treatment effects on advanced math course-taking in the Norway study, it was important to confirm that challenge-seeking, as assessed by our task, was, in fact, associated with students’ course-taking decisions in the Norwegian context. The continuous make-a-math worksheet measure significantly predicted a student’s likelihood of being enrolled in advanced math, b = .016, SE = .002, t = 8.00, p < .001. This validity analysis therefore supports the interpretation that course-taking is a challenge-seeking measure.

Next, the Norway study replicated the treatment effects on advanced math course-taking previously seen in the U.S data. The average effect, ignoring the moderation by school opportunity, was 3 percentage points, from 46% of students taking the theoretical math class in the control condition to 49% in the growth mindset condition, SE = .010, N = 6,541, t = 2.85, p = .005. A 3 percentage point treatment effect (and a 7% relative increase from the baseline of 46%) is noteworthy because (a) it is the same point estimate reported in the NSLM (Yeager et al., 2019), (b) it represents about as large of an effect that could be hoped for by Kahneman (Dubner, 2017); and (c) this effect size was obtained from a low-cost and scalable intervention in a population-generalizable sample, on an outcome with known consequences for students’ economic trajectories. Moreover, when we consider whether the environment allowed students to act on any increased desire for challenge, the effect was even larger.

An analysis of whether students took and passed the more challenging math class at the end of the year showed the same effect size of 3 percentage points, SE = .009, N = 6,541, t = 2.82, p = .006. This is important because it shows that more students could have been receiving advanced training in mathematics than were previously, but their mindset or motivation may have been holding them back. Because Norwegian students must pass standardized assessments of their theoretical math knowledge in order to pass the course, the present results are the best evidence to date that a growth mindset does not simply lead to differences in motivation, but can also lead to greater knowledge.

Research Question 3: Mindset × Context Interaction

When schools offered students the choice of advanced vs. non-advanced math several months after the treatment sessions (i.e. post-treatment choice schools)—that is, when school structures afforded students the opportunity for their greater motivation to learn to translate into more rigorous course-taking—there was a 6 percentage-point effect of the intervention, SE = .014, z = 4.24, p < .001, which is a 10% increase relative to the base rate of 60% among controls.² When students had already chosen their math course prior to receiving the intervention (i.e. pre-treatment choice schools), and therefore when students needed to navigate tacit, bureaucratic rules to alter their course-taking, there was, surprisingly, a 2 percentage point benefit of the intervention SE = .010, t = 1.82, p = .069, which is a 5% increase relative to a base rate of 44% among controls. Because the intervention was positive (albeit small and imprecisely estimated) even in pre-treatment choice schools, the Growth mindset × Context (1 = pre-treatment, 0 = post-treatment) interaction was small but nevertheless statistically significant b = .04, SE = .016, t = 2.63, p = .010.

GENERAL DISCUSSION

How can we inspire more adolescents to be “learners”—to seek out intellectual challenges that will create new expertise, even when doing so may be difficult or unpleasant? This is a matter of global importance. The economy of the future will require individuals to opt in to training programs, both in person and online, and choose to persist in those programs even when doing so is difficult. Moreover, the math skills that may be gained by challenge seeking—and the technical skills in science that they unlock—are critical building blocks for valuable human capital (National Research Council, 2012).

The present study demonstrates that psychological science has something to say about this pressing issue. A growth mindset intervention, teaching that intellectual abilities can be developed, and why people want to develop them, increased adolescents’ willingness to take on intellectual challenges in math in two nations. The effects emerged on a behavioral marker of challenge-seeking—the make-a-math-worksheet task—across student demographic groups and achievement levels, and in generalizable samples. High- and low-achieving students alike sought out challenges when treated, somewhat in contrast to past studies that have used grade point averages as the primary outcome and shown benefits among students who did not already have high grades (Paunesku et al., 2015; Yeager et al., 2016, 2019).

The Norway experiment also showed that the willingness to be a learner can translate into entry into course pathways that build stronger skills, provided that schools made it easy for students to move up. This finding supports Mindset × Context Theory and aligns with a core Lewinian (1952) insight that adjustments to psychology do not change behavior in isolation, but depend on and interact with the affordances of a context (also see Ferrer & Cohen, 2019; Walton & Yeager, in press).

Interpreting Effect Sizes

Experts in public health and epidemiology have argued for decades that treatments with small effects on average for individuals can have substantial effects for populations, provided that the interventions can be scaled up (see Greenberg & Abenavoli, 2017). And recall that the present growth mindset treatment showed a 3 percentage-point effect overall and 6 percentage points when policies opened the gateway to advanced math. What’s more, students actually passed those classes, which means they ended up with more knowledge. Considering the long-term benefits to health, wealth, and well-being of advanced math (Carroll et al., 2017), there is potential for the intervention to be used in combination with efforts to increase access to rigorous learning opportunities and improve human capital at a population scale.

Implications and Future Directions

These results have broad implications for psychological science and growth mindset in particular. A program of research stretching from laboratory studies to field studies resulted in an intervention approach that changed behavior in the most rigorous kind of design: a random-assignment experiment conducted in a random sample of schools (US Study) or nearly all schools in two counties (Norway Study), with independent data collection, a pre-registered analysis plan (US Study) and a replication and extension by economists with no stake in mindset research but strong expertise in behavioral experiments (Norway Study). It is noteworthy that we replicated the NSLM’s effect on advanced math course-taking because that was the “exploratory” outcome reported by Yeager et al. (2019), and sometimes exploratory analyses have been accorded less validity. Yet, reassuringly, the intervention altered the outcome of course-taking months later, with the exact same point estimate for the average treatment effect. In a scientific climate that is questioning the validity and replicability of basic insights from psychological science, these findings are a reminder that the slow and careful path from basic science to replication and application remains viable.

Even so, the present research spotlights the fact that replication studies should continue pay attention to local context, both in the design of the intervention (Yeager et al., 2016) and in the analysis of potential interactions with school contexts. Although growth mindset intervention effects replicated in two large randomized trials, that does not mean that growth mindset will always have the same effects in every study, or that we can ignore other sources of heterogeneity in the future, such as fidelity to study protocols or peer norms.

We note that a brief, self-administered online intervention is not the only (or even the best) means for creating a growth mindset. The present study used a direct-to-student method (rather than classroom or school-level treatment) primarily because it gave us clean, person-level random assignment, which provided high statistical power, and because it did not require specialized training of adults, which can be resource-intensive. Larger effects might have been obtained from a more intensive and elaborate treatment that involved face-to-face interactions (Blackwell et al., 2007), or that was delivered in concert with a new and challenging curriculum (Andersen & Nielsen, 2016). Of course, more elaborate treatments may be more difficult to scale up, and would need to retain an autonomy-supportive tone (rather than a didactic, “preachy” tone) if they are to be effective—a tone that can be challenging to maintain in lengthier interventions (see Yeager et al., 2018).

The present treatment did not attempt to change teachers’ practices, such as how they praise or challenge students or whether and how they provide feedback on assignments. In part this is because teacher professional development has a poor track record of effectiveness in large districts (TNTP, 2015), and has proven difficult to scale with fidelity. That does not mean that changing teacher behavior is unimportant, however. It is the most critical next step. New research is showing that classrooms and schools already communicate mindset messages (Hooper et al., 2016; Kraft & Grace, 2016), and when they happen to communicate growth mindset messages students outperform their prior grades and test scores (Canning et al., 2019; Hooper et al. 2016). Ongoing research into the transmission of mindsets through teachers may lead to innovations about how to craft mindset-supportive environments that create more robust benefits for students (see Haimovitz & Dweck, 2017).

Conclusion

On the basis of the present evaluation, high schools that are interested in promoting greater challenge-seeking and reducing fixed mindsets would be justified in implementing the growth mindset intervention (the U.S. intervention is available to schools at no cost to them via www.perts.net). Regardless, the present results reinforce the need to find ways to alleviate the fears that individuals face as they confront intellectual challenges. How might we craft classrooms and workplaces that communicate that challenge is a route to learning, rather than something that makes people feel as though they are not talented enough? And what about home environments or online learning environments? The future of mindset science has much to learn about—and hopefully much to contribute to—these questions of great importance.

Public Significance Statement.

The success of the global economy of the future depends on people’s willingness to be “learners”—that is, their motivation to seek out challenging training experiences that develop new, valuable skills. Here we show that a short, scalable growth mindset intervention lasting under an hour increased high school students’ willingness to be “learners” in a random sample of U.S. schools and with nearly all academic-track schools in two counties of Norway. Critically, however, the online intervention was not a magic bullet; schools’ course enrollment policies, which governed how easy or hard it was to enter advanced coursework, moderated the long-term effects of the short intervention.