Reducing Inequality in Academic Success for Incoming College Students: A Randomized Trial of Growth Mindset and Belonging Interventions

Abstract

Light-touch social psychological interventions have gained considerable attention for their potential to improve academic outcomes for underrepresented and/or disadvantaged students in postsecondary education. While findings from previous interventions have demonstrated positive effects for racial and ethnic minority and first-generation students in small samples, few interventions have been implemented at a larger scale with more heterogeneous student populations. To address this research gap, 7,686 students, representing more than 90% of incoming first-year students at a large Midwestern public university, were randomly assigned to an online growth mindset intervention, social belonging intervention, or a comparison group. Results suggest that after the fall semester, the growth mindset intervention significantly improved grade point averages for Latino/a students by about .40 points. This represents a 72% reduction in the GPA gap between White and Latino/a students. Further, this effect was replicated for both spring semester GPA and cumulative GPA. These findings indicate that light-touch interventions may be a minimally invasive approach to improving academic outcomes for underrepresented students. Our findings also highlight the complexity of implementing customized belonging interventions in heterogeneous contexts.

Introduction

In 2014, nearly 18 million undergraduate students were enrolled in two- or four-year institutions (Ginder, Kelly-Reid, & Mann, 2015). This number represents an increase of approximately 35% since 2000 (Snyder & Dillow, 2015). As enrollment rates have dramatically increased, the percentage of students receiving their bachelor’s degree within a six-year period has also modestly increased, from 52% of students in the 1996 starting cohort, to 56.5% of students in the 2006 starting cohort (Snyder & Dillow, 2015). However, this rate of degree completion continues to vary widely across student subgroups. The challenge of completing a four-year degree appears to be most acute for low-income, racial minority, and first-generation students, who have lower rates of persistence and completion in four-year colleges than their higher income, White, and continuing-generation counterparts (Bailey & Dynarski, 2011; Bowen, Chingos, & McPherson, 2009; Ifill et al., 2016).

Faced with these challenges, policymakers in higher education have begun to look for ways to reduce inequality between the rates at which advantaged and disadvantaged students persist in college and complete advanced degrees (Executive Office of the President, 2014). The possible solutions are numerous and diffuse; however, social-psychological interventions have recently gained attention as a “light-touch” approach for increasing motivation for persistence (Yeager & Walton, 2011). These interventions target college students’ uncertainty about belonging (i.e., beliefs about encountering commonplace adversities while trying to “fit in” in a new community) or growth mindsets (i.e., the belief that ability can be grown by exerting effort, seeking help from others, and revising strategies in the face of challenge; Yeager & Dweck, 2012) to help students overcome barriers to success. Such interventions may reduce responsiveness to negative events, build confidence to handle daily stressors, and/or lead students to adopt a mindset that views intelligence as malleable and open to growth, as opposed to fixed and immutable (Wilson, 2006; Wilson, Damiani, & Shelton, 2002; Yeager & Walton, 2011).

Given that light-touch interventions are relatively brief and efficient to implement and have shown positive effects for disadvantaged students in several secondary and postsecondary contexts (Yeager et al., 2016), it is important to know how they perform when carried out in a new context by a new research team. If these interventions could improve academic outcomes for a disadvantaged student subgroup, at little to no cost, they would be tremendously beneficial to both students and universities, especially given the high cost of recruiting, retaining, and remediating students who fall behind. However, rigorous trials and replications are necessary to examine whether the effects seen in earlier work remain robust when implemented in new settings by new researchers.

One such initiative is the Spartan Persistence Program (SPP), a social-psychological intervention designed to enhance student efficacy and sense of belonging based closely on prior studies shown to have positive effects for subgroups of disadvantaged students (Yeager et al., 2016). To assess the effectiveness of this intervention, a block randomized controlled trial was implemented in summer 2014 with the incoming student population at Michigan State University (MSU). The following study examines the effect of this intervention on the academic outcomes of underrepresented student subgroups in the first-year class during their first two semesters.

Light-Touch Psychological Interventions

Over the past several years, psychologists have conducted a series of light-touch social-psychological interventions in universities to enhance student performance and completion (Yeager et al., 2016). These interventions are specifically designed to benefit disadvantaged students, such as first-generation college students or members of racial/ethnic minority groups that are often underrepresented relative to White, continuing-generation students in four-year colleges and universities. Disadvantaged students often face unique challenges related to postsecondary success, such as implicit signals that they may be viewed as less able or not talented enough to succeed (Olson & Dweck, 2008). If students enter college aware of these signals, this may lead them to interpret everyday challenges in ways that might confirm their worries. The goal of these interventions is to change these signals from being interpreted as evidence of subgroup-specific inadequacy to a manifestation of broader challenges faced by incoming students in general, thereby reducing students’ uncertainty about their own ability and sense of belonging.

One of these types of interventions is the “growth mindset” intervention, which aims to shift the way in which students attribute academic success or failure from stable factors (typically one’s fixed intelligence) to more unstable factors (e.g. effort or social conditions). In other words, they aim to convince students that rather than being fixed and finite, intelligence is malleable, and one can become smarter and more successful in school by working harder (Yeager & Walton, 2011). For example, Blackwell, Trzesniewski, and Dweck (2007) administered a growth mindset intervention with a sample of 91 low-income, racial/ethnic minority students in an urban middle school. The treatment consisted of a series of eight weekly sessions in which students learned about the function of the brain and how the brain could become stronger by taking on challenges. The intervention resulted in positive effects of about .30 grade points for students in the treatment condition compared to students in a control condition.

This type of intervention has also been replicated in postsecondary contexts. For example, Aronson, Fried, and Good (2002) developed an intervention based on a “pen pal” session in which college students wrote letters to middle school students explaining and endorsing the concept of malleable intelligence. In a sample of 79 students at a selective university, they found that the intervention increased grade-point averages in the following academic term by .23 points. They also found that the intervention increased African American students’ engagement and identification with the school.

A second class of light-touch intervention blends work on attribution and implicit theories of intelligence (Wilson & Linville, 1982, 1985) with that of stereotype threat (Steele & Aronson, 1995). These interventions, referred to here as belonging uncertainty interventions, aim to help disadvantaged students reframe worries they may have about fitting in as normal, rather than as reinforcement of societal and institutional signals that they do not belong or are unable to succeed. For example, Walton and Cohen (2007, 2011) implemented a social belonging intervention with 92 first-year students at a selective college. The intervention was delivered through a one-hour laboratory session, which included reviewing the results of a survey that suggested that many students feel like they do not belong at first, but these worries fade over time. Students also wrote essays and gave speeches for the purpose of explaining to future students that worries about fitting in change as time goes on. Results found that the intervention significantly improved the GPAs of African American students by .24 points relative to African American students in the control condition. This effect persisted from sophomore to senior year, and represented a 52% reduction in the GPA gap between African American and White students.

The success of light-touch interventions in small samples and laboratory settings has necessitated further application of these methods in larger, more diverse contexts and settings. Paunesku et al., (2015) used online modules to administer growth mindset and sense-of-purpose interventions in a sample of 1,594 students in thirteen high schools, and found that students who were designated at risk (those with a GPA less than 2, or who had failed at least one course) and received the growth mindset intervention, the sense-of-purpose intervention, or a double dose of the two earned GPAs about .13 points higher than at-risk students in the control condition. This design and approach was important for several reasons. First, it applied growth mindset interventions on a relatively large scale using online modules, as opposed to more time-consuming and intensive in-person applications of the treatment. This study served as a test of more large-scale use of this type of social-psychological intervention across schools or districts. Second, the intervention was applied to a heterogeneous sample of high school students, so this intervention provided evidence as to the efficacy of growth mindset interventions when delivered in more heterogeneous samples.

Recently, both growth mindset and belonging uncertainty interventions have been implemented at large scale as a more general treatment for incoming first-year college students. For example, Yeager et al. (2016), Study 2 used online modules to administer growth mindset and social belonging interventions with a sample of 7,335 incoming students at a large, flagship state university. This study found that both the growth mindset and social belonging interventions significantly impacted academic outcomes for disadvantaged students. In this case, disadvantaged students were classified as racial minority or first-generation students. Low graduation rates of previous cohorts of these students led the authors to conclude that these groups of students are known to be disadvantaged relative to their majority peers in terms of completing college. Specifically, 73% of disadvantaged students who received either intervention completed their first year as full-time enrolled students, compared to 69% of disadvantaged students in a control condition. This difference represented a 40% reduction in the inequality between full-time enrollment rates between disadvantaged and advantaged students. This study demonstrated the efficacy of large-scale application of growth mindset and social belonging interventions in a postsecondary setting.

Yeager et al. (2016), Study 3 also conducted a similar intervention with 1,592 incoming first-year students at a selective private university. Here, students were randomly assigned to receive one of three lay-theory interventions designed to impact social belonging, cultural fit, or perception of critical feedback. Interventions were administered online as a part of the university’s prematriculation orientation process. Results indicated that disadvantaged students who received any of the lay theory interventions ended the year with cumulative GPAs about .09 points higher than disadvantaged students in a control group. In this context, based on historical student enrollment and persistence data and relevant psychological theory, all African American, Latino, Native, Pacific Islander, and first-generation European American students were classified as disadvantaged. Consistent with theory and prior results, no significant effects were found for advantaged students, in this case continuing-generation European American students and all Asian students. This impact translated into a 31% adjustment in the raw achievement gap between advantaged and disadvantaged students, and a 47% reduction after adjusting for covariates.

The initial success of these recent interventions has emphasized the need for additional applications of light-touch social-psychological interventions in more varied and diverse educational contexts. As Paunesku et al. (2015) suggest, social-psychological interventions need to be scalable if they are to impact students beyond those in tightly controlled research settings. However, this raises the issue of customization. Often, social-psychological interventions have been customized for each intervention site, and their effectiveness may depend upon how well the intervention fits the context of the site (Yeager & Walton, 2011). This is especially true for belonging interventions. Although mindset interventions may be effective when using largely standardized materials (Paunesku et al., 2015), social belonging interventions (e.g. Yeager et al., 2016) likely need to be customized to appropriately address the social challenges faced by a particular group in a particular context. Balancing the need for customization with the need for scalability is a challenge that will require researchers to design light-touch social-psychological interventions that both broadly anticipate the needs of disadvantaged students and are modifiable to local contexts. The SPP furthers these ambitions, with the broader aim of repeating the study for multiple cohorts and following the persistence of the effects over time. As a large, public land-grant university, with significant racial, ethnic, and socioeconomic diversity, Michigan State is an important context in which to study the efficacy of light-touch interventions. This study describes the findings from the first cohort of the SPP.

Research Questions

In this study we examine the following research questions:

1. Did participation in either an online growth mindset or social belonging intervention cause incoming undergraduate students at MSU from underrepresented racial/ethnic groups to earn higher GPAs, attempt more course credits, or complete more course credits compared to their peers in the control group after completing their first two semesters?

2. For students in underrepresented groups, was the impact of these interventions moderated by any background characteristics, including high school GPA, ACT score, Pell eligibility, first-generation status, and pre-intervention levels of growth mindset and belonging uncertainty?

Method

Over the last five years, motivated by a desire to increase retention and graduation rates, Michigan State University has undertaken a series of systematic reforms to increase students’ engagement with the university and improve their sense of belonging as part of the campus community. Previous intervention work (e.g., Yeager & Walton, 2011) caught the attention of university administrators, who then convened a committee of expert faculty to design a similar set of interventions specific to MSU students. A pilot study of about 1,000 incoming students was conducted in January 2014; this facilitated the development of randomization procedures as well as refinement and improvement of the two treatment conditions and the comparison condition.

Procedures and Sample

Prior to fall enrollment, incoming first-year students at MSU are required to attend a two-day summer orientation program. The summer orientation includes information about course enrollment, academic programs, social and cultural resources, and other key features of campus life. Several weeks prior to their orientation session, students were sent a link from a university officer to an online survey on Qualtrics requesting their participation. Students were permitted to complete the survey any time before their scheduled orientation session; those who did not complete the survey before orientation were given time to do so after arriving. After providing several pieces of demographic information, students were randomized (blocking on race/ethnicity using the standard Qualtrics randomizer) into one of three conditions: (a) a mindset condition, (b) a belonging condition, and (c) a control condition.

Eligible participants for this study were identified by their participation in MSU’s Academic Orientation Program in Summer 2014, which is required for all incoming students with “Freshman” academic status. In total, 8,331 students were scheduled to participate in summer orientation and were therefore eligible for participation in the study; of those 8,331 students, 7,686 responded to the invitation to participate for a response rate of 92%. After completing the online intervention materials, all participants were asked a series of post-intervention questions that were used to both confirm the validity of the interventions and gauge students’ engagement with them. All other academic outcomes used here were measured and collected by the University Registrar at the end of the fall semester.

Since the intervention took place at one site, randomization occurred at the individual level, within separate blocks according to the students’ racial/ethnic group. The blocks, which correspond to the university’s internal schema for reporting race and ethnicity, were as follows: White, Latino/a,¹ African American, Asian, and Multiracial. International students were blocked separately but also received both interventions. Given that the interventions were designed to impact underrepresented student groups, we focus our reporting primarily on results for Latino/a and African American students, with results for White students included as a comparison. An analysis of MSU’s graduation statistics over the past decade indicate that both Latino/a students and African American students were both consistently underrepresented in the initial composition of each incoming student class relative to their distribution in the state of Michigan K–12 student population, and also significantly less likely to persist and complete an undergraduate degree within six years relative to their White and Asian peers. International students were excluded from this analysis, resulting in a final analytic sample of 6,529 students (Mindset n = 2,135; Belonging n = 2,172; Control n = 2,222).²

Growth Mindset Intervention Condition

Students in the mindset intervention group read a short scientific article on “Building the Brain” that introduced the concept of brain plasticity, or the idea that the brain, similar to other muscles, can grow when given repeated practice in the face of challenges (Yeager & Dweck, 2012). The purpose behind this article is to expose students to the idea that their intelligence is not fixed, and that extra effort and focus on their part can translate to significant growth in intelligence over time. It also argues that instead of just a “knowing” part of the brain there is a “know how” part of the brain that also can improve with time (Yeager & Dweck, 2012). After reading the brief article, students are asked several reflective questions in which they are encouraged to identify moments in their own lives when they may have (or have not) adopted a growth mindset. Students are encouraged to write open-length responses to each reflective question, including writing a piece of advice for a future first-year student based on lessons learned from the article. Responses varied widely, but a typical response read like this:

In high school the muscles in your brain were geared for less “weight.” Basically meaning you were really well developed mentally for that type of learning environment. In college you are going to have to lift a lot more weight. You are going to be required to ask a lot of questions and learn a lot of lessons on your own. This is okay, but this can cause a lot of frustration. Just remember to never give in and reach out. In college you are going to be challenged to know how to use the information and learn the skills. The best things to remember are: Be brave and reach out, don’t give up, and don’t get discouraged!

After completing the preliminary demographic portion of the Qualtrics survey, students typically spent between twenty and twenty-five minutes on the mindset intervention activities.

Social Belonging Intervention Condition

Distinct from the mindset treatment, where a fixed set of information was being presented to students for their reflection, students in the belonging treatment group were given a series of stories ostensibly taken from the responses of upperclassmen on a recent survey investigating the challenges of starting out in college. These stories dealt with a series of issues around leaving home, including finding friends, homesickness, fitting in socially, and trying to find one’s identity as a member of a new community. The stories were carefully developed from a series of focus groups with a diverse set of MSU juniors and seniors, in which facilitators discussed how current students overcame challenges to feeling like they belonged at MSU, what belonging at MSU means, etc. Quotes from these focus groups served as source material for the stories presented to new students during summer orientation. Each story the student reads is attributed to an upperclassman at MSU given a pseudonym. Further, the first story the student reads is matched with the reader’s identified gender and race/ethnicity. In other words, if the student is female and African American, the first story presented is from the perspective of an African American female. Later stories in the series were from other racial/ethnic and gender pairings. After reading the stories, students were then asked to reflect on their meaning for their own lives in a series of short reflective responses. Student responses again varied significantly, but several typical responses include:

Student #1:

At first I wasn’t excited to attend college because I like having a group of friends and family that I already know at home. As college started approaching I got more comfortable with the idea because I realized that everyone is in the same boat as me. Now, I am excited and ready to meet new people and to be a Spartan.

Student #2:

I am worried that it will not be easy for me to make friends. I’m worried that it be hard to be accepted in such a big environment. I worry that I will get to college and realize that I am not cut out for the classes and I will have a hard time adjusting and finding people who I can relate to. However, coming from a big high school, I’m confident that I will eventually find friends, I just worry that when I get there, people will already know each other. I would like to be able to join clubs and be an active part of Michigan State University. I’m excited to be a part of something that I can truly say I am proud to be a part of. I’m excited to experiences all the things that MSU has to offer and I can’t wait to get past the bumpy first part of the transition to school, and begin my life at MSU!

Students typically spent between fifteen and twenty minutes completing the belonging treatment activities.

Control Condition

The control condition took the form of a placebo and was the same control used in previous similar research (Walton & Cohen, 2011), with small modifications to fit the context at MSU. It was designed to give students the impression that the institution was providing help, but in contrast to the social belonging condition, which focused on students’ feelings of uncertainty about fitting in the social environment, the control condition focused on changes in the physical environment. For example, stories were included that talked about the weather in East Lansing, adjusting to a new class schedule, finding your way at such a large campus, and finding places to eat. The control condition was designed to be relatively inanimate, in contrast to the more personal stories of the social belonging treatment condition. As with the belonging intervention, students in the comparison condition were given a series of stories to read and then asked to reflect on what they mean for them as they begin their college experience. Students typically spent between ten and fifteen minutes on the control condition materials. It is important to note that given the slight difference in orientation between the mindset and belonging conditions, the control is a better match with the belonging treatment and can be thought of as a control group for belonging and a comparison group for mindset.

Data Sources

For the analyses presented below, we used data from four sources: (a) pre-intervention online surveys, (b) post-intervention online surveys, (c) the university registrar, and (d) the university office of institutional research. For our investigation, we used several student background measures collected by the university, including high school GPA, ACT/SAT scores, Pell grant eligibility, and first-generation status. The university registrar was the primary source for all outcome measures used in the study.

Outcome Variables

This study uses five primary outcome measures, all of which are strongly associated with a student’s persistence to a second year of college and eventual completion of a BA. All outcomes were obtained via the Office of the Registrar at the completion of the fall 2014 semester and the spring 2015 semester. The first outcome, grade point average (GPA), is calculated in the conventional format, by multiplying the numerical course grade (ranging from 0–4, in increments of 0.50) by the number of credits for a given course, totaling the grade points, and then dividing by the number of credits taken for the semester. The second outcome, course credits attempted, is the number of total credits a student attempted in each semester. Recognizing that the definition and measurement of attempted credits may vary by institution, we adhered to MSU’s definition of credits attempted, which is measured as the number of credits the student took by the end of the semester, after drops and withdrawals. Course credits completed is the total number of credits for which a student received a passing grade (in this case 1.0, or a “D”). Full-time enrollment is defined as attempting 12 or more credits during the fall semester. Finally, cumulative GPA is calculated as a credit-weighted average of fall and spring semester GPAs.

Covariates

Several instruments were used on the pre-intervention survey to gauge both the balance of the randomization between treatment and control groups, as well as controls in the intent-to-treat regression estimates for both interventions. These scales include the prospective belonging uncertainty scale used in Yeager et al. (2016), Study 2, originally adapted from Walton and Cohen (2011). For this measure, students responded to four questions, each on a scale from 1 (Not at all true) to 5 (Completely true). The separate items were as follows:

1. Sometimes I worry that I will not belong in college.

2. I am anxious that I will not fit in at college.

3. I feel confident that I will belong in college. (reverse-coded)

4. When I face difficulties in high school, I wonder if I will really fit in when I get to college.

Student item scores demonstrated sufficient reliability (Cronbach’s α = .82), and were combined into a composite measure that represented the arithmetic mean of all four responses.³

A second instrument on the pre-intervention survey was used to measure students’ initial levels of growth mindset. Here we used the three-question scale developed by Hong, Chiu, Dweck, Lin, and Wan (1999). For this measure, students responded to three questions, each on a scale from 1 (Strongly disagree) to 6 (Strongly agree). The separate items were as follows:

1. You have a certain amount of intelligence, and you really can’t do much to change it. (reverse-coded)

2. Your intelligence is something about you that you can’t change very much. (reverse-coded)

3. You can learn new things, but you can’t really change your basic intelligence. (reverse-coded)

As with the belonging uncertainty scale, student item scores on the pre-intervention growth mindset scale demonstrated sufficient reliability (Cronbach’s α = .81) and were combined into a composite measure that represented the arithmetic mean of all three responses.

Several additional variables were included as controls in the regression estimates of the treatment effect. These included ACT score, high school GPA, Pell grant eligibility, and first-generation status. For the small proportion of students (less than 15%) who only reported SAT scores, these were converted to ACT scores using the standard concordance published by the college board (2009). Reported high school GPAs above 4.0 were capped at 4 to account for weighted and unweighted GPAs. Pell eligibility was operationalized as a binary measure, equal to one if the student was classified as Pell eligible, and zero if otherwise. First-generation status was also operationalized as a binary measure, equal to one if the student reported being first in their family to attend college, and zero if otherwise. All additional control variables were provided by MSU’s institutional research office.

Tests for Equivalence of Experimental Groups

To assess whether the randomization procedures resulted in a sufficient balance in baseline measures between the three experimental groups, sample means were compared for each treatment group relative to the control condition using independent samples t tests for continuous measures and two-sample proportion tests for binary measures. Results are presented in Table 1. No significant differences were found on any baseline measures between the two intervention groups and the control group.⁴

Table 1.

Effectiveness of random assignment at Time 1, by experimental group.

		Belonging experiment			Mindset experiment
	Overall mean	Belonging mean	Control mean	p value (T – C)	mindset mean	Control mean	p value (T – C)
ACT score	25.60	25.66	25.56	.38	25.59	25.56	.82
High school GPA	3.65	3.65	3.65	.92	3.65	3.65	.83
Belonging uncertainty	3.88	3.88	3.87	.85	3.90	3.87	.35
Growth Mindset	4.76	4.77	4.77	.89	4.74	4.77	.21
Female	0.54	0.55	0.53	.21	0.54	0.53	.48
First generation	0.24	0.24	0.24	.74	0.24	0.24	.97
Pell eligible	0.27	0.28	0.25	.06	0.27	0.25	.27
n	6,529	2,172	2,222	—	2,135	2,222	—

Note. Binary variables tested with two-sample proportion test and scale variables tested with two-sample t test (H₀: T – C = 0).

Attrition from the study was closely monitored and assessed in two forms: (a) overall attrition, which is the percentage of the overall sample (both treatment and control) that is lost from the initial randomization, and (b) differential attrition, which is the extent to which attrition occurs in an unbalanced fashion, with higher rates in one group than another. Table 2 presents sample sizes and attrition rates for both the mindset and belonging interventions. In both cases, overall attrition was less than 3%. The same is true for differential attrition, which was less than 1% in both situations. Comparing these rates to established guidelines from the Institute for Education Sciences (2014a, 2014b), these rates of attrition fall well within “tolerable” range for obtaining relatively unbiased treatment effect estimates.

Table 2.

Sample sizes and attrition rates for mindset and belonging experiments.

Mindset experiment
Time 1		Time 2		Attrition rates
N Treat	N Control	N Treat	N Control	Overall	Differential
2,135	2,222	2,068	2,161	2.90%	0.40%
Belonging experiment
Time 1		Time 2		Attrition rates
N Treat	N Control	N Treat	N Control	Overall	Differential
2,172	2,222	2,116	2,161	2.70%	0.20%

Notes. Control group (belonging control) is same for both experiments. Time 1 = end of fall semester, Time 2 = end of spring semester.

Overall, rates of missing data in this study were low. For the pre-randomization questions used to examine the balance of treatment and control groups, rates of missingness were less than 1%. This was also the case for the four initial outcomes (GPA, credits attempted, credits completed, and full-time enrollment), and ACT composite score, which all had rates of missingness below 1%. Somewhat higher rates of missingness occurred for high school GPA, which was obtained from a separate administrative data collection. Overall, 7% of observations were missing high school GPA, but this rate did not differ significantly between the treatment and control conditions.

Given that students were randomly assigned to these groups and that rates of missingness do not differ, this suggests that the possible bias due to differential rates of missing responses is negligible and that missing data could likely be classified as missing completely at random (MCAR) (Rubin, 1987). If data truly are MCAR, missing observations can be deleted listwise and complete case analysis should provide unbiased estimates. However, given that the MCAR assumption is typically unprovable, we chose to adopt a more rigorous missing data approach that could meet the missing at random standard (MAR). Here, missingness is only dependent upon observed variables. To account for missing data in the regression estimation of treatment effects, we used multiple imputation to simulate 50 additional data sets with plausible values for missing entries. The procedure was performed using the mi impute mvn routine in Stata (StataCorp, 2015) which uses multivariate normal regression, paired with an iterative Markov chain Monte Carlo simulation process to perform data augmentation (Little & Rubin, 2002; Schafer, 1997). Estimates are then performed on each augmented data set and pooled to provide a single estimate. For comparison, Tables A1 and A2 in the appendix provide treatment effect estimates using listwise deletion, multiple imputation, and the dummy-variable method of accounting for missing data (Cohen, Cohen, West, & Aiken, 2002; Puma, Olsen, Bell, & Price, 2009).⁵ Overall, the magnitude and significance of estimates did not vary by missing data approach.

Results

First, we present initial results of each intervention on academic outcomes using independent samples t tests. Next, we present intent-to-treat estimates of each intervention’s impact of academic outcomes using multiple linear regression. Finally, we explore whether any baseline or demographic characteristics moderated the impact of the growth mindset intervention. Moderator analysis of the social belonging intervention was excluded from this report because the intervention was not found to have a significant impact in this sample.⁶ For all analyses, results are presented by subgroup, with separate impact estimates for Latino/a students, African American students, and White students. Results for the full sample, including students from all racial/ethnic groups, are included for reference.

Initial Results by Subgroup

Table 3 compares outcomes for the growth mindset intervention by subgroup using independent samples t tests. Latino/a students in the mindset treatment group had significantly higher GPAs for both the fall semester (${\overline{x}}_T=3.13$, ${\overline{x}}_C=2.73$, t = 3.16, d = .46) and the spring semester (${\overline{x}}_T=2.97$, ${\overline{x}}_C=2.64$, t = 2.27, d = .33), as well as a higher cumulative GPA after their first year of classes (${\overline{x}}_T=3.05$, ${\overline{x}}_C=2.69$, t = 2.92, d = .42). No significant differences in academic outcomes were found for African American students or for White students. Table 4 presents results for the social belonging intervention. Here, across all three subgroups and the full sample, no significant differences in academic outcomes were found between students in the treatment and control conditions.

Table 3.

Treatment estimates for growth mindset intervention, by subgroup.

	Treatment group		Control group
Outcome	M	SD	M	SD	t/z	Cohen’s d
Latino/a students (n = 193)
Fall semester
GPA	3.13	0.79	2.73	0.94	3.16**	.46
Credits completed	13.14	2.54	12.44	3.29	1.63	.24
Credits attempted	13.66	1.75	13.67	1.45	−0.06	−.01
Full-time enrollment	0.97	—	0.97	—	−0.01	−.003
Spring semester
GPA	2.97	0.91	2.64	1.10	2.27*	.33
Credits completed	12.80	3.69	11.87	4.16	1.65+	.24
Credits attempted	13.13	3.51	13.02	3.42	0.21	.03
Cumulative GPA	3.05	0.88	2.69	0.96	2.92**	.42
African American students (n = 318)
Fall semester
GPA	2.60	0.91	2.56	0.86	0.43	.05
Credits completed	11.87	3.26	12.06	2.83	−0.57	−.06
Credits attempted	13.20	1.50	13.21	1.43	−0.05	−.01
Full-time enrollment	0.93	—	0.94	—	−0.52	−.06
Spring semester
GPA	2.37	1.01	2.41	0.93	−0.38	−.04
Credits completed	11.15	4.04	11.43	3.54	−0.66	−.07
Credits attempted	12.62	2.86	12.91	2.21	−0.99	−.11
Cumulative GPA	2.48	0.82	2.48	0.79	0.002	.0002
White students (n = 3,416)
Fall semester
GPA	3.24	0.70	3.22	0.72	0.81	.03
Credits completed	13.29	2.15	13.29	2.28	0.04	.001
Credits attempted	13.58	1.68	13.60	1.75	−0.32	−.01
Full-time enrollment	0.96	—	0.96	—	−0.23	−.01
Spring semester
GPA	3.14	0.88	3.17	0.87	−0.99	−.03
Credits completed	13.07	3.23	13.14	3.16	−0.61	−.02
Credits attempted	13.46	2.86	13.46	2.82	−0.01	−.0003
Cumulative GPA	3.19	0.71	3.20	0.73	−0.19	−.01
All students (n = 4,357)
Fall semester
GPA	3.18	0.75	3.14	0.78	1.57	.05
Credits completed	13.18	2.32	13.12	2.46	0.84	.03
Credits attempted	13.57	1.69	13.55	1.73	0.28	.01
Full-time enrollment	0.96	—	0.96	—	−0.35	−.01
Spring semester
GPA	3.06	0.93	3.08	0.92	−0.66	−.02
Credits completed	12.87	3.43	12.91	3.32	−0.42	−.01
Credits attempted	13.34	2.99	13.37	2.85	−0.33	−.01
Cumulative GPA	3.12	0.77	3.11	0.78	0.39	.01

Notes. For continuous outcomes (GPA, credits attempted, and credits completed), independent samples t tests were used, and t values are presented. For full-time enrollment, a binary outcome, a two-sample proportion test was used, and z scores are presented.

⁺p < .10.

^*p < .05.

^**p < .01.

Table 4.

Treatment estimates for belonging uncertainty intervention, by subgroup.

	Treatment group		Control group
Outcome	M	SD	M	SD	t/z	Cohen’s d
Latino/a students (n = 192)
Fall semester
GPA	2.92	0.99	2.73	0.94	1.31	.19
Credits completed	12.36	3.33	12.44	3.29	−0.18	−.03
Credits attempted	13.13	2.70	13.67	1.45	−1.74	−.25
Full-time enrollment	0.94	—	0.97	—	−1.06	−.18
Spring semester
GPA	2.68	1.20	2.64	1.10	0.20	.03
Credits completed	11.84	4.62	11.87	4.16	−0.04	−.01
Credits attempted	12.80	3.69	13.02	3.42	−0.43	−.06
Cumulative GPA	2.80	0.98	2.69	0.96	0.77	.11
African American students (n = 336)
Fall semester
GPA	2.65	0.96	2.56	0.86	0.93	.10
Credits completed	11.76	3.08	12.06	2.83	−0.93	.10
Credits attempted	13.02	1.38	13.21	1.43	−1.26	−.14
Full-time enrollment	0.94	—	0.94	—	−0.27	−.03
Spring semester
GPA	2.41	1.07	2.41	0.93	0.02	.002
Credits completed	11.11	4.12	11.43	3.54	−0.77	−.08
Credits attempted	12.49	3.07	12.91	2.21	−1.41	−.15
Cumulative GPA	2.53	0.92	2.48	0.79	0.51	.06
White students (n = 3,437)
Fall semester
GPA	3.23	0.72	3.22	0.72	0.20	.01
Credits completed	13.26	2.37	13.28	2.28	−0.30	−.01
Credits attempted	13.56	1.91	13.60	1.75	−0.54	−.02
Full-time enrollment	0.95	—	0.96	—	−1.09	−.04
Spring semester
GPA	3.15	0.91	3.17	0.87	−0.62	.02
Credits completed	13.02	3.33	13.14	3.16	−1.14	.04
Credits attempted	13.42	2.87	13.46	2.82	−0.49	.02
Cumulative GPA	3.19	0.74	3.20	0.73	0.26	.01
All students (n = 4,394)
Fall semester
GPA	3.16	0.77	3.14	0.78	0.82	.02
Credits completed	13.08	2.50	13.12	2.46	−0.45	−.01
Credits attempted	13.49	1.88	13.55	1.73	−1.14	−.03
Full-time enrollment	0.95	—	0.96	—	−1.30	−.05
Spring semester
GPA	3.06	0.96	3.08	0.92	−0.76	−.02
Credits completed	12.79	3.50	12.91	3.32	−1.17	−.04
Credits attempted	13.32	2.90	13.37	2.85	−0.56	−.02
Cumulative GPA	3.11	0.79	3.11	0.78	−0.03	−.001

Notes. For continuous outcomes (GPA, credits attempted, and credits completed), independent samples t tests were used, and t values are presented. For full-time enrollment, a binary outcome, a two-sample proportion test was used, and z scores are presented.

Estimated Subgroup Impacts of Growth Mindset and Social Belonging Interventions on Academic Outcomes

In Table 5, we present the results of three separate OLS regression models and one logistic regression model that were used to estimate the intent-to-treat impacts of the growth mindset intervention on participants’ fall and spring semester outcomes. To estimate the causal impact of the treatment condition, for each academic outcome, we fit a regression with the outcome as the dependent variable and an indicator for the treatment group as a predictor. All models also control for a series of baseline covariates, including initial belonging uncertainty, initial growth mindset, ACT score, high school GPA, first-generation status, and Pell grant eligibility. As with the previous t test analysis, regression models were estimated by subgroup, with relevant estimates presented for Latino/a students, African American students, and White students. Estimates for the full sample are also provided for reference.

Table 5.

Summary of treatment effects using multiple imputation for growth mindset intervention, by race/ethnicity subgroup.

	Latino/a students (n = 193)			African American students (n = 318)			White students (n = 3,416)			All students (n = 4,357)
Outcome	B	SE	p value	B	SE	p value	B	SE	p value	B	SE	p value
Fall semester
GPA	0.38	0.11	.001	0.03	0.09	.78	0.02	0.02	.39	0.03	0.02	.10
Credits completed	0.65	0.40	.11	−0.24	0.33	.46	0.002	0.07	.98	0.06	0.07	.39
Credits attempted	−0.01	0.25	.98	0.02	0.16	.90	−0.02	0.06	.72	0.01	0.05	.79
Full-time enrollment	0.76	0.83	.74	0.86	0.48	.75	0.95	0.18	.79	0.95	0.15	.73
Spring semester
GPA	0.33	0.14	.02	−0.05	0.10	.60	−0.03	0.03	.32	−0.02	0.03	.44
Credits completed	0.89	0.54	.10	−0.32	0.42	.44	−0.07	0.11	.55	−0.04	0.10	.67
Credits attempted	0.08	0.48	.86	−0.31	0.28	.28	−0.004	0.10	.97	−0.03	0.09	.71
Cumulative GPA	0.35	0.11	.002	−0.01	0.09	.87	−0.004	0.02	.86	0.01	0.02	.73

Notes. For GPA, credits attempted, and credits completed, treatment effects were estimated by subgroup using separate OLS regressions. In each regression model, in addition to the treatment indicators, the following additional controls were included: initial belonging uncertainty, initial growth mindset, ACT score, high school GPA, first-generation status (first gen = 1), and Pell eligibility (Pell = 1). For full-time enrollment, a logistic regression model was used with the same covariates. Coefficients presented here are unstandardized regression coefficients for GPA, credits attempted, and credits completed. For full-time enrollment, treatment effects are presented as odds ratios comparing the treatment condition to the control condition.

Results indicated that relative to their counterparts in the control condition, Latino/a students in the growth mindset condition earned higher GPAs during the fall semester (B = 0.38, p = .001), as well as during the spring semester (B = 0.33, p = .02), and they also had higher cumulative GPAs after the completion of the full academic year (B = 0.35, p = .002). No significant effects were found for course credits attempted, completed, or full-time enrollment for Latino/a students. For African American students, no effects were found for course credits attempted, GPA, credits completed, or full-time enrollment. As expected, given theory and prior research, the growth mindset intervention had no significant effect for White students. Intent-to-treat impacts were also estimated for the social belonging intervention, and followed the same modeling procedures described above. Results can be found in Table 6. No significant treatment effects were found across all three subgroups or the full sample.

Table 6.

Summary of treatment effects using multiple imputation for belonging uncertainty intervention, by race/ethnicity subgroup.

	Latino/a students (n = 192)			African American students (n = 336)			White students (n = 3,437)			All students (n = 4,394)
Outcome	B	SE	p value	B	SE	p value	B	SE	p value	B	SE	p value
Fall semester
GPA	0.21	0.13	.10	0.03	0.10	.77	0.01	0.02	.77	0.02	0.02	.40
Credits completed	−0.003	0.47	0.99	−0.47	0.32	.15	−0.02	0.08	.76	−0.03	0.07	.66
Credits attempted	−0.52	0.34	.13	−0.23	0.15	.14	−0.04	0.06	.55	−0.07	0.05	.23
Full-time enrollment	0.31	.74	.12	0.90	0.47	.83	0.82	0.17	.27	0.82	0.15	.18
Spring semester
GPA	0.02	0.15	.91	−0.09	0.10	.39	−0.02	0.03	.59	−0.02	0.03	.44
Credits completed	−0.14	0.62	.81	−0.57	0.41	.17	−0.13	0.11	.25	−0.04	0.10	.67
Credits attempted	−0.31	0.53	.56	−0.50	0.32	.12	−0.06	0.10	.57	−0.03	0.09	.71
Cumulative GPA	0.11	0.13	.38	−0.03	0.09	.73	−0.004	0.02	.87	−0.002	0.02	.92

Notes. For GPA, credits attempted, and credits completed, treatment effects were estimated by subgroup using separate OLS regressions. In each regression model, in addition to the treatment indicators, the following additional controls were included: initial belonging uncertainty, initial growth mindset, ACT score, high school GPA, first-generation status (first gen = 1), and Pell eligibility (Pell = 1). For full-time enrollment, a logistic regression model was used with the same covariates. Coefficients presented here are unstandardized regression coefficients for GPA, credits attempted, and credits completed. For full-time enrollment, treatment effects are presented as odds ratios comparing the treatment condition to the control condition.

Interaction Effects Between Baseline Characteristics and Treatment Assignment

To further explore the relationship between treatment assignment and students’ baseline characteristics, we conducted moderator analyses to see if the impact of the treatment varied according to students’ ACT score, high school GPA, gender, first-generation status, Pell grant eligibility status, or pre-intervention levels of growth mindset and belonging uncertainty. Results are found below in Table 7. Since significant treatment effects were found only for fall, spring, and cumulative GPA for Latino/a students in the growth mindset condition, only these estimates are presented, along with those of African American students and White students in the growth mindset condition for comparison.

Table 7.

Results from separate linear regressions testing for moderation of mindset intervention, MI estimates.

	Latino/a students (n = 193)			African American students (n = 318)			White students (n = 3,416)
Moderator tested	B	t	p value	B	t	p value	B	t	p value
Outcome: Fall GPA
ACT score	−0.06	−1.97	.05	−0.02	−0.75	.45	0.01	1.06	.29
High school GPA	−0.56	−1.49	.14	−0.34	−1.26	.21	−0.03	−0.36	.72
Belonging uncertainty	0.02	0.11	.92	−0.04	−0.34	.74	0.02	0.54	.59
Growth mindset	0.23	1.53	.13	−0.27	−2.42	.02	−0.01	−0.48	.63
Gender (female = 1)	0.18	0.78	.44	−0.17	−0.83	.41	−0.05	−1.04	.30
First-gen. status	0.10	0.43	.66	−0.22	−1.18	.24	0.03	0.47	.64
Pell eligibility	0.25	0.24	.29	−0.29	−1.33	.18	−0.01	−0.14	.89
Outcome: Spring GPA
ACT score	−0.05	−1.43	.15	−0.03	−0.84	.40	0.01	0.61	.54
High school GPA	−1.05	−2.30	.02	−0.38	−1.31	.19	0.07	0.07	.94
Belonging uncertainty	−0.33	−1.80	.07	0.01	0.08	.94	0.04	1.15	.25
Growth mindset	−0.05	−0.27	.79	−0.14	−1.12	.26	−0.001	−0.05	.96
Gender (female = 1)	0.24	0.85	.40	0.14	0.65	.51	−0.04	−0.53	.60
First-gen. status	−0.23	−0.80	.43	−0.20	−0.97	.33	0.08	1.15	.25
Pell eligibility	0.24	0.84	.40	−0.07	−0.30	.77	0.08	1.14	.25
Outcome: Cumulative GPA
ACT score	−0.06	−1.84	.07	−0.03	−0.91	.37	0.01	0.91	.36
High school GPA	−0.80	−2.14	.03	−0.36	−1.47	.14	−0.01	−0.14	.89
Belonging uncertainty	−0.16	−1.02	.31	−0.01	−0.14	.89	0.03	0.97	.33
Growth mindset	0.09	0.59	.55	−0.21	−1.99	.05	−0.01	−0.29	.77
Gender (female = 1)	0.21	0.90	.37	−0.01	−0.06	.95	−0.04	−0.82	.41
First-gen. status	−0.06	−0.26	.79	−0.21	−1.23	.22	0.05	0.93	.35
Pell eligibility	0.25	1.03	.30	−0.18	−0.90	.37	0.04	0.66	.51

Notes. Each row represents the test of a mindset intervention × moderator interaction in a linear regression model with Fall GPA as the outcome. Regressions were run by subgroup, and included the mindset intervention condition variable, the moderator, and ACT score and high school GPA as covariates. All moderators were centered.

For Latino/a students, high school GPA was a negative and significant moderator of the relationship between treatment assignment and spring semester GPA (B = −1.05, p = .02) as well as between treatment assignment and cumulative GPA (B = −0.80, p = .03), which suggests that the growth mindset intervention may be less effective for students with higher high school GPAs. Also for Latino/a students, ACT score was a negative and significant moderator of the relationship between treatment assignment and fall semester GPA (B = −0.06, p = .05), which suggests that the mindset intervention may be less effective for students with higher ACT scores. For African American students, initial levels of growth mindset interacted with the treatment assignment when predicting fall semester GPA (B = −0.27, p = .02) as well as cumulative GPA (B = −0.21, p = .05), which suggests that the growth mindset treatment may be less beneficial for students with higher baseline growth mindset beliefs. No moderators were significant in the White student sample.

Sensitivity Analysis

To assess the sensitivity of the treatment effect on GPA found for Latino/a students in the mindset treatment group, we performed several additional analyses. Applying Frank, Maroulis, Duong, and Kelcey (2013) replacement of cases approach to quantifying robustness, we find that to invalidate the effect of the mindset intervention, 43% of the cases (or 83 Latino/a students) in the mindset treatment group would have to be replaced with cases for which the mindset treatment has zero effect. Further, we assessed the potential that an omitted variable might impact the results, and find that an omitted variable would have to be correlated at 0.347 with GPA and at 0.347 with the treatment indicator (conditioning on observed covariates) to invalidate an inference. Correspondingly the impact of an omitted variable (as defined in Frank, 2000) must be (0.320 × 0.320) = 0.12 to invalidate an inference. This impact is more than ten times larger than the impact of the largest observed covariate, the pre-measure of growth mindset.⁷ Given that the randomization achieved balance between treatment and control groups on all observed covariates, the possibility of identifying an omitted variable of that strength that correlates highly with the treatment assignment is unlikely.

Discussion

The purpose of the present study was to test whether participation in either an online growth mindset or social belonging intervention led disadvantaged incoming first-year students to earn higher GPAs, attempt more course credits, or complete more courses over their first two semesters of college than their peers in the control group. Further, the study tested whether the impact of these interventions varied according to students’ background characteristics. We found that Latino/a students who received the growth mindset intervention had significantly higher first-semester GPAs than did their peers in the control group. This difference was about 0.40 grade points in a raw comparison of growth mindset and control groups, and about .38 points after adjusting for covariates. This represents a large and significant difference, both in terms of effect size (Cohen’s d = .46), and in terms of practical significance. Latino/a students in the control group finished the fall semester with an average GPA of 2.73, whereas Latino/a students in the growth mindset intervention finished with an average GPA of 3.13. Given MSU’s grading scale, this means moving from an average grade of C+/B− to an average grade of B. This could likely have real positive implications for students’ progress through their degree program and eventual degree completion, as well as postgraduate plans such as admission into graduate school. Further, given the GPA of White students in the control group (3.20), the effects seen here from the growth mindset intervention are equivalent to a 72% reduction in the GPA gap between Latino/a and White students.

Further, the GPA effect observed for Latino/a students in the mindset group persisted into the spring semester, where Latino/a participants in the mindset group earned GPAs that were .33 points higher than participants in the control group (Cohen’s d = .33). The same trend was observed in cumulative GPA, where Latino/a participants in the mindset group ended the year with cumulative GPAs that were .36 points higher than participants in the control group (Cohen’s d = .42). These findings were robust both to the type of missing data approach used, as well as to the inclusion of academic and demographic covariates in multiple regression models.

Findings from this study support the notion that growth mindset interventions can improve academic outcomes for disadvantaged students (Paunesku et al., 2015; Yeager et al., 2016), specifically improving students’ GPAs. Interestingly, not all groups of disadvantaged students in this study seemed to benefit equally, as African American students in the mindset intervention did not report higher GPAs. Further, this study did not find an effect for the mindset intervention on full-time enrollment, which was a key finding by Yeager et al. (2016), Study 2, who found a 4% increase in full-time enrollment for disadvantaged students who received a growth mindset treatment relative to their peers in the control condition.

The question remains as to why the growth mindset interventions may have had differential effects for Latino/a students even when compared to other disadvantaged groups, such as African American students. One possibility is that Latino/a students’ levels of academic preparation led them to be particularly sensitive to the growth mindset treatment. Compared as a block, Latino/a students in this sample had significantly higher high school GPAs and ACT scores than African American students.⁸ This suggests that the mindset intervention may be more beneficial for students with somewhat higher levels of academic preparation (or a combination of higher academic preparation and lower levels of growth mindset; see below). Additionally, results from the moderator analysis found that the growth mindset intervention was disproportionately beneficial to Latino/a students with lower ACT scores (for the fall GPA outcome) or lower high school GPAs (for the spring and cumulative GPA outcomes). This was not the case for African American or White students, which suggests additional differences in the way Latino/a students’ initial academic preparation may have interacted with the mindset intervention. In all cases, Pell eligibility was not found to be a significant moderator, which suggests that this was not a characteristic that led to effect heterogeneity.

A second possibility may be related to students’ pretreatment levels of growth mindset. Latino/a students overall were found to have significantly lower levels of pre-intervention growth mindset compared to African American students.⁹ Further, results from the moderator analysis found that initial levels of growth mindset were a significant and negative moderator of the growth mindset treatment effect for African American students for both fall and cumulative GPA. In other words, African American students with higher initial levels of growth mindset saw less impact from the growth mindset intervention. This moderation was not present in the Latino/a subgroup—the treatment effect was not moderated by initial growth mindset. If Latino/a students across the distribution of prior growth mindset had relatively homogenous treatment impacts, while African American students experienced heterogeneous impacts, with lower effects for students with higher levels of growth mindset, this could possibly explain why the intervention might have had differential effects for one disadvantaged group relative to the other.

Evidence of heterogeneous effects by race/ethnicity in similar light-tough social-psychological interventions is quite limited. Typically, studies have included comparisons of advantaged and disadvantaged students in larger blocks (e.g., comparing White and Asian continuing-generation students to all African American, Latino/a, and White first-generation students), which is useful (and congruent with the psychological theory underpinning these interventions), but does not explore possible heterogeneity by race/ethnicity. In this study, separate randomization was conducted by blocking by race/ethnicity in an effort to build larger samples of underrepresented students over time. This facilitated comparisons by race/ethnic block, which also revealed these differences in effect. Further research in this area could benefit from more intentional exploration of these differences by race/ethnic group and the possible explanations thereof. However, we suggest that these differences may also be driven by systematic differences in levels of pre-intervention growth mindset and academic preparation between African American and Latino/a students.

Further, in contrast to previous findings with incoming first-year students at a flagship state university (Yeager et al., 2016, Study 2), our social belonging intervention did not yield significant results on any outcome measure. Both Study 2 and Study 3 in Yeager et al. (2016)) reported significant impacts from a social belonging intervention. Study 2, conducted at a flagship state university, reported a treatment impact on full-time enrollment of 13% from the social belonging treatment. Study 3, conducted at a selective private university, reported a treatment effect of .06 points in cumulative first-year GPA. This may suggest institutional differences between the sites where these studies were conducted, or it may also be a result of the customization needed to calibrate the social belonging intervention to Michigan State University.

Given the specificity of the social belonging intervention to the physical and social context in which it is applied, it is possible that the design of the intervention may be more sensitive to “misfit,” which may reduce the impact of the intervention overall. It is also possible that the implementation of the intervention within the larger context of a two-day orientation program may have changed the nature of the control condition, providing participants with an experience that may address similar uncertainties to those addressed by the treatment, and thereby reducing the expected treatment effect. This raises important questions about what it might mean to properly calibrate an intervention such as this to a particular context and may require additional modification and innovation, including a shift in the timing of the intervention to after the initial orientation period.

Limitations

Several limitations of the current study merit further attention. One important limitation of this study is the relatively narrow window of time between administration of the intervention and measurement of fall and spring semester academic outcomes. This study demonstrates the impact of interventions after one academic year; additional waves of data collection will be needed to examine whether the positive impacts seen here for Latino/a students persist for more distal outcomes such as college completion. In addition, Latino/a students, which were the only subgroup that demonstrated positive impacts from the mindset intervention, comprise only 5% of the larger student sample. This may limit the generalizability of the findings to a wider population of Latino/a undergraduates in other regional or institutional contexts.

Further, for purposes of estimating treatment effects on academic outcomes, this study does not differentiate between different types of course credit attempted and completed. Prior research, such as Yeager et al. (2016), Study 2, examined the impact of similar interventions on full-time enrollment, but restricted their analysis to students’ enrollment in core academic courses, as opposed to elective courses (e.g., physical education). This decision was justified by an extensive analysis of factors of student success that found full-time enrollment in core classes to be the strongest predictor of eventual college completion. In this study, given the constraints of our available data, all course credits were included, not just those attempted or completed in core academic classes.

Conclusion

Taken in context with the results of other studies implementing social-psychological interventions (e.g., Paunesku et al., 2015; Yeager et al., 2016), the results of our study suggest that light-touch interventions can improve disadvantaged students’ college GPA, though they are far from the only worthwhile intervention. However, given the low cost and relatively simple implementation of light-touch interventions, they could be more readily applied to a wider population of disadvantaged students. Although promising, they also offer unique implementation challenges. As Yeager and Walton (2011) suggest, these interventions aren’t magic, and require careful customization to institutional context. More research is needed into the mechanisms that underlie light-touch interventions and how they might be systematically modified and customized to meet a range of different strategic purposes. In sum, light-touch interventions may not be a universal solution to the inequities many students face today, but they can be one of an array of targeted tools and interventions that can be used to promote equity in postsecondary education.

Appendix

Table A1.

Comparison of estimated mindset treatment effect coefficients, by race/ethnic subgroup and missing data technique.

	Latino/a students (n = 193)			African American students (n = 318)			White students (n = 3,416)			All students (n = 4,357)
Outcome	MI	DV	LD	MI	DV	LD	MI	DV	LD	MI	DV	LD
Fall semester
GPA	0.38***	0.36***	0.38***	0.03	0.04	0.07	0.02	0.02	0.01	0.03	0.03+	0.03
Credits completed	0.65	0.58	0.78+	−0.24	−0.49	−.01	0.002	0.02	−0.06	0.06	0.06	0.03
Credits attempted	−0.01	−0.06	0.08	0.02	0.03	0.10	−0.02	−0.01	−0.04	0.01	0.02	−0.04
Full-time enrollment	0.76	0.41	0.79	0.86	0.84	0.87	0.95	0.97	0.75	0.95	0.95	0.77
Spring semester
GPA	0.33*	0.31*	0.39*	−0.05	−0.05	.003	−0.03	−0.03	−0.02	−0.02	−0.02	−.001
Credits completed	0.89+	0.84	1.02+	−0.32	−0.29	0.09	−0.07	−0.05	−0.03	−0.04	−0.04	0.03
Credits attempted	0.08	0.04	0.21	−0.31	−0.31	−0.06	−0.004	0.01	0.05	−0.03	−0.03	0.04
Cumulative GPA	0.35**	0.34**	0.38**	−0.01	−.002	0.04	−0.004	−.002	−0.01	0.01	0.01	0.01

Notes. For GPA, credits attempted, and credits completed, treatment effects were estimated by subgroup using separate OLS regressions. In each regression model, in addition to the treatment indicators, the following additional controls were included: survey response rate, initial belonging uncertainty, initial growth mindset, ACT score, high school GPA, and mother’s education (BA or higher = 1). For full-time enrollment, a logistic regression model was used with the same covariates. Coefficients presented here are unstandardized regression coefficients for GPA, credits attempted, and credits completed. For full-time enrollment, treatment effects are presented as odds ratios comparing the treatment condition to the control condition. MI = multiple imputation; DV = dummy variable method; LD = listwise deletion.

⁺p < .10.

^*p < .05.

^**p < .01.

^***p < .001.

Table A2.

Comparison of estimated belonging uncertainty treatment effect coefficients, by race/ethnic subgroup and missing data technique.

	Latino/a students (n = 209)			African American students (n = 361)			White students (n = 3,611)			All students (n = 4,657)
Outcome	MI	DV	LD	MI	DV	LD	MI	DV	LD	MI	DV	DL
Fall semester
GPA	0.21+	0.19	0.35**	0.03	0.03	0.01	0.01	0.01	0.02	0.02	0.02	0.03
Credits completed	−0.003	−0.1	0.52	−0.47	−0.49	−0.58+	−0.02	−0.01	−0.02	−0.03	−0.03	−0.04
Credits attempted	−0.52	−0.59+	0.08	−0.23	−0.23	−0.19	−0.04	−0.02	−0.04	−0.07	−0.06	−0.06
Full-time enrollment	0.31	0.32	2.75	0.90	0.91	0.99	0.82	0.85	0.72	0.82	0.83	0.77
Spring semester
GPA	0.02	−0.002	0.15	−0.09	−0.09	−0.13	−0.02	−0.02	0.002	−0.02	−0.02	−0.01
Credits completed	−0.14	−0.23	0.17	−0.57	−0.57	−0.57	−0.13	−0.12	−0.02	−0.04	−0.12	−0.04
Credits attempted	−0.31	−0.37	−0.08	−0.50	−0.51	−0.46	−0.06	−0.04	0.07	−0.03	−0.05	0.03
Cumulative GPA	0.11	0.09	0.25*	−0.03	−0.03	−0.06	−0.004	−0.003	0.01	−0.002	−0.003	0.01

Notes. For GPA, credits attempted, and credits completed, treatment effects were estimated by subgroup using separate OLS regressions. In each regression model, in addition to the treatment indicators, the following additional controls were included: survey response rate, initial belonging uncertainty, initial growth mindset, ACT score, high school GPA, and mother’s education (BA or higher = 1). For full-time enrollment, a logistic regression model was used with the same covariates. Coefficients presented here are unstandardized regression coefficients for GPA, credits attempted, and credits completed. For full-time enrollment, treatment effects are presented as odds ratios comparing the treatment condition to the control condition. MI = multiple imputation; DV = dummy variable method; LD = listwise deletion.

⁺p < .10.

^*p < .05.

^**p < .01.