Skip to main content
PMC home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2021 Jan 8.
Published in final edited form as: Educ Res Eval. 2020 Jan 31;25(5-6):248–269. doi: 10.1080/13803611.2020.1717542

An exploration of engagement and effectiveness of an online values affirmation

Benjamin L Bayly a,*, Matthew F Bumpus b
PMCID: PMC7793545  NIHMSID: NIHMS1551822  PMID: 33424419

Abstract

Brief social-psychological interventions, like the values affirmation (VA), that target individual feelings of competency and buffer against social threats, have been shown to effectively reduce achievement gaps in randomised controlled trials. In the current study, underrepresented minority and first-generation college students in their first university semester (N = 496) were randomly assigned to receive the VA electronically or complete an online survey (control). Results revealed: (a) VA participants did not engage with the intervention in a manner typical of past VA studies that delivered the intervention as a class activity; (b) VA students had lower semester grade point averages (GPAs) than control students; and (c) contrary to previous studies, neither stereotype threat nor social belonging moderated the effectiveness of the VA. These findings further emphasise the importance of the context within which the VA is delivered and highlight the challenges that accompany increasing the reach of the VA through a widespread, online delivery.

Keywords: Values affirmation, brief interventions, participant engagement, underrepresented college students, online interventions

Introduction

Over the past decade, non-intrusive brief social-psychological interventions targeting individual thoughts and beliefs have gained popularity due to positive findings from randomised controlled trials (RCT; Yeager & Walton, 2011). These interventions are unique in that they do not directly target behaviours, but instead are intended to promote feelings of competency, reaffirm self-worth, and buffer against social threats. With an increase in attention stemming from the efficacy of these interventions, a desire to scale up the interventions to reach more individuals has followed. However, Yeager and Walton (2011) caution against the assumption that brief psychological interventions are straightforward and will be effective regardless of the setting and instead argue that they are largely context dependent.

Values affirmation

One brief intervention that has generated encouraging results is the values affirmation (VA; Steele & Liu, 1983). The VA is designed to assist individuals in overcoming perceived stressful environments by buffering against social threats through bolstering self-affirmation (Harackiewicz et al., 2014). In this intervention, individuals are instructed to select the two or three values that are the most important to them and spend 10 min reflecting and writing about why these values are important (Fein & Spencer, 1997; Sherman, Nelson, & Steele, 2000). When tested as an RCT, control-group participants are instructed to select values that are unimportant to them and write about why these values could be important to someone else. VA interventions have generated encouraging results as demonstrated by a recent meta-analysis indicating an average effect size of d = .38 on academic outcomes (Lazowski & Hulleman, 2016).

Academic achievement gaps

Academic performance and, more specifically, academic achievement gaps, is a common outcome targeted by the VA. For example, Miyake and colleagues (2010) found that the VA reduced the discrepancy in grades between men and women by 61% in an introductory physics class. Additionally, the VA has shown to be effective in a pair of RCTs among middle-school African American students, where African American students in the treatment groups performed significantly better academically than African American students in the control groups (Cohen, Garcia, Apfel, & Master, 2006). As a result of the VA, the achievement gap between White/European American and African American students was reduced by 40%. Recently, the VA has been used to target academic achievement gaps between first-generation (i.e., students whose parents did not graduate from college) and continuing-generation university students. For example, in a double-blind study, the VA intervention resulted in significantly higher biology grades and overall grade point average (GPA) when comparing treatment and control groups among first-generation students (Harackiewciz et al., 2014).

Theoretical background of the intervention

The VA is hypothesised to be effective through reducing stress and reaffirming feelings of self-worth in situations that are challenging or threatening to an individual’s confidence (McQueen & Klein, 2006; Sherman & Cohen, 2006). The intervention was initially designed as a mechanism to bolster self-esteem (Steele & Liu, 1983); later, Steele and Aronson (1995) delivered the intervention with marginalised groups to combat stereotype threat. Recent studies have examined threats to social belonging and employed a cultural mismatch theoretical background as one possible explanation for why the intervention may be effective for participants who are prone to question if they belong in a certain environment (e.g., Harackiewicz et al., 2014; Layous et al., 2017).

Stereotype threat

Stereotype threat arises when a member of a group feels additional stress in an environment due to a negative stereotype about the group (Steele & Aronson, 1995). In these situations, the individual is aware of the negative stereotype and feels pressure to perform in a way to rebuke the stereotype (Steele & Aronson, 1995). The VA is thought to buffer against stereotype threat and reaffirm feelings of self-worth (Cohen et al., 2006). Consistent with this rationale, the VA was particularly beneficial for women in a physics course who endorsed the stereotype that women were less capable than men in science, technology, engineering, and maths (STEM) fields (Miyake et al., 2010). The authors concluded that the endorsement of this stereotype was detrimental for women in the control group, but the relationship between stereotype endorsement and academic performance in the treatment group was eliminated. Similarly, researchers found the VA intervention to be effective for women facing stereotype threat when performing a maths laboratory task (Martens, Johns, Greenberg, & Schimel, 2006). In one group, participants were told the test was diagnostic of their maths ability, thus activating stereotype threat, and in the other group participants were told the test had nothing to do with their maths ability. For women in the stereotype threat group, participants who did not receive the VA intervention performed statistically worse than women receiving the VA or women in the non-stereotype threat group. In the non-stereotype group, there was not a significant difference between women who received the intervention and those who did not, suggesting the intervention was only effective for those encountering stereotype threat (Martens et al., 2006). Further supporting the stereotype threat rationale is the fact that the VA intervention has consistently been ineffective for groups not encountering negative stereotypes about their performance (Cohen et al., 2006; Harackiewicz et al., 2014; Martens et al., 2006; Miyake et al., 2010).

Cultural mismatch theory

Cultural mismatch theory (Stephens, Townsend, Markus, & Phillips, 2012) suggests that individual performance is often influenced by whether there is alignment between individual cultural norms and the established norms at the macrolevel. This theory is relevant for first-generation students as universities in the US often emphasise independent values (e.g., achievement), whereas first-generation students tend to endorse interdependent values (e.g., connection) to a greater extent than continuing-generation students (Stephens et al., 2012). Although cultural mismatch theory is not as common a rationale in the VA literature as stereotype threat, there is emerging evidence to suggest that it is relevant for the effectiveness of the VA. Tibbetts and colleagues (2016) found the VA was most effective in targeting academic achievement among first-generation students for students who wrote about independence. Additionally, first-generation students who wrote about independence in a laboratory setting performed better on a standardised assessment (Tibbetts et al., 2016). Taken together, these results suggest that mechanisms to promote alignment of values between first-generation students and the broader context around them may be effective in improving academic achievement.

Experiencing cultural mismatch often leads to diminished feelings of social belonging (Stephens et al., 2012), but the VA intervention may buffer against this threat. In a study where the VA was administered among seventh-grade students, a content analysis was conducted for the written responses to the values prompt. The authors found that African American middle schoolers typically valued social belonging (Shnabel, Purdie-Vaughns, Cook, Garcia, & Cohen, 2013). The authors then delivered the VA to college students targeting a gender-achievement gap in maths tests, but instead of having only treatment and control groups, two additional groups were added. One group was instructed to write about why their values made them feel self-sufficient (i.e., the independent group), and the other group wrote about why their values made them feel connected with others (i.e., the social-belonging group). While the VA was successful in eliminating the gender achievement gap, women in the social-belonging group significantly outperformed men (Shnabel et al., 2013). While this study did not directly target cultural mismatch, it does provide preliminary evidence that reinforcing feelings of social belonging may ultimately benefit individuals who may otherwise be more likely to question if they belong in a specific environment.

Linking stereotype threat and social belonging.

Stereotype threat, cultural mismatch, and social belonging are not mutually exclusive constructs. A wealth of research has demonstrated that awareness of negative stereotypes leads to diminished feelings of social belonging and cues that elicit uncertainty in belonging often trigger stereotype threat (see Walton & Carr, 2012). Essentially, if an individual is in a threatening environment where they perceive negative opinions towards them, they may question whether they belong in that environment. Furthermore, if an individual feels they do not belong in a situation they may attribute the cause to negative stereotypes. Additionally, Walton and Cohen (2011) designed an intervention to promote social belonging through presenting social adversity as something that is normative and temporary. The intervention was particularly effective for African American students; the intervention group reported significantly less attention to negative racial stereotypes, which coincided with stronger feelings of social belonging (Walton & Cohen, 2011). This suggests that feelings of belonging and stereotype threat are linked for underrepresented groups. The VA intervention may ultimately be promoting feelings of students’ social belonging by serving as a buffer against stereotype threat.

Who benefits from the VA intervention?

It is important to recognise that although there is a body of evidence demonstrating the utility of the VA intervention, other studies have failed to replicate positive results (e.g., Dee, 2015; Hanselman, Rozek, Grigg, & Boman, 2017; Harackiewicz, Canning, Tibbetts, Priniski, & Hyde, 2016; Kost-Smith et al., 2012). One possible explanation for the null findings from these studies is that the participants were in an environment they perceived as less threatening than participants in other VA studies. This would align with a previous study where the VA was shown to be ineffective in high schools where the majority of students were underrepresented minorities (URM; Bratter, Rowley, & Chukhray, 2016). This rationale is also supported by Hanselman, Bruch, Gamoran, and Borman (2014), who found the VA was more effective for URM middle-school students in schools with lower percentages of URM students. Additionally, the theoretical perspective behind the VA suggests the level of psychological threat in a given environment is relevant for the effectiveness of the intervention. The theory behind the intervention is to buffer against a psychological threat (e.g., stereotype threat) and as such is not intended to benefit those who are not facing a psychological threat (Yeager & Walton, 2011).

Area of opportunity

VA interventions have typically been delivered in classrooms during regularly scheduled class time (e.g., Cohen et al., 2006; Harackiewicz et al., 2014) or in supervised research laboratory settings (e.g., Creswell et al., 2005). Additionally, in VA studies that have examined academic achievement (i.e., GPA) as an outcome, the VA has typically been delivered as a class activity or assignment (e.g., Harackiewicz et al., 2014; Miyake et al., 2010). A handful of VA interventions have been delivered online, but these were still delivered with a connection to a specific course (e.g., Jordt et al., 2017; Kizilcec, Saltarelli, Reich, & Cohen, 2017).

An effective online VA intervention delivered outside of a class assignment presents an opportunity to dramatically increase the reach of the intervention (Griffiths, Lindenmeyer, Powell, Lowe, & Thorogood, 2006). Additionally, because the VA is designed to buffer against stereotype threat and social belonging uncertainty, an online VA carries important implications as it represents a mechanism to reach students who may avoid attending class as a way to cope with these social threats. Given these potential benefits, it is important to assess if an electronic version of the intervention is an appropriate method to increase the reach of the VA.

Research questions and hypotheses

Our aim in this study was to examine if students would engage with an electronic VA when presented outside a class activity and evaluate the effectiveness of the VA. We developed three research questions (RQ), one exploratory and two with accompanying hypotheses (H).

Exploratory RQ: Will students engage with the online VA intervention protocol delivered outside of a homework assignment and not tied to a specific course?

RQ1: Is an online delivery of the VA intervention effective for:

  1. URM university students?

  2. First-generation university students?

H1a. URM students in the treatment group will have significantly higher semester GPAs than URM students in the control group.

H1b. First-generation students in the treatment group will have significantly higher semester GPAs than first-generation students in the control group.

RQ2: What variables moderate the effect of the VA intervention?

H2a. The relationship between the VA and GPA will be moderated by stereotype threat for URM students. The VA will be more effective for URM students who report higher levels of baseline stereotype threat.

H2b. The relationship between the VA and GPA will be moderated by social belonging. The intervention will be more effective for students who report lower baseline levels of social belonging.

Method

Participants

URM and first-generation students entering their first year of college were recruited from a large, residential, public university in the western US through the university’s Office of Multicultural Student Services (MSS). We used MSS’s definition of both URM and first-generation students; MSS defines URM students as those who do not identify as European American/White and first-generation students as students who do not have a parent that graduated from a university with a bachelor’s degree. MSS oversees multiple student programmes designed to facilitate a positive university experience among multicultural, first-generation, and other underrepresented students. All URM students have the opportunity to receive tutoring and mentoring from MSS, and the centre makes a concentrated effort to encourage students to build social connections with their peers through clubs and events hosted by MSS. First-generation students who are not URM are eligible to receive assistance from MSS if they are deemed to be at a higher academic risk when entering their first year at the university. All URM and first-generation students for whom MSS had contact information (1,720 students) were contacted at the beginning of the second week of the semester and at the beginning of Week 7 (around midterm exams). These time points were chosen given prior research indicating that the beginning of the semester and the week of midterm exams are periods of increased stress for students (Bland, Melton, Welle, & Bingham, 2012). Unless students chose to opt out when they were initially contacted, they received an email at both time points. At both time points, students received an email at the start of the week, a reminder email at the end of the week, and a final reminder email at the start of the next week. Participants were entered to win $50 gift cards at both time points.

Randomisation to condition was done before the initial email. We were interested in exploring mediational effects of the VA for a separate research study, and, as such, we chose to oversample the treatment group in an effort to ensure an adequately sized treatment group to test indirect effects (20% assigned to control; 80% assigned to treatment; indirect results are not reported in the current study). Table 1 shows the sample size per condition at both time points. In all, 496 students participated (control n =107, response rate = 31%; treatment n = 389, response rate = 28%). The sample was racially/ethnically diverse: 41.6% Chicano/Latinx, 20.0% Asian American, 18.1% Biracial (non-Latinx), 7.6% European American/White, 8.5% African American, 2.2% Native American, and 2.0% Pacific Islander. Consistent with the university MSS definition of URM, all students who were not European American/White were considered URM. Participant race and ethnicity was dramatically different from the first-year university population as a whole (61.4% European American/White, 16% Chicano/Latinx). Our sample was primarily female (69.6%) and first-generation students (56%). No continuing-generation, non-URM students were contacted to participate in the study (URM first-generation students n = 222; URM continuing-generation students n = 181; non-URM first-generation students n = 21; remaining 72 were missing either first-generation or URM status).

Table 1.

Sample size by condition at each time point.

Control Treatment
Time 1 n = 70 n = 230
Time 2 n = 68 n = 238
Times 1 and 2 n = 31 n = 79
Total n = 107 n = 389
Dose Within Treatment Group
Selected Values Completed Reflection
Time 1 n = 181 n = 165
Time 2 n = 181 n = 160
Times 1 and 2 n = 49 n = 43
Total n = 313 n = 282
Open in a new tab

Note: All participants who completed the reflection also selected values.

Procedure

The VA protocol employed in this study was similar to other VA interventions (e.g., Harackiewicz et al., 2014), which has been validated through prior research (Cohen, Aronson, & Steele, 2000). At the start of the second week of the fall semester (participants’ first semester at the university), all students received an email from MSS inviting them to participate in an online survey; we did not describe the study as an “intervention” in recruitment materials or in the survey. We chose to have the invitation email appear to come from MSS as we felt this would increase the number of students who responded, as MSS was likely to be an organisation that they had some level of familiarity with. MSS administrators supported this student recruitment procedure. In the invitation email from MSS, all participants were informed that there were no right or wrong answers and that their responses would be valuable for understanding what first-year students typically encounter during their first year at the university. Control participants were instructed that the survey would take approximately 5 to 10 min and included questions related to how they feel in the university environment, their motivations for attending college, and how they managed judgemental thoughts. VA participants were given similar instructions in the invitation email but were also informed that they would be asked to select values from a list and spend 10 min writing about why these values were important to them. The VA intervention was included after the survey questions. VA participants were given instructions for completing the activity and were again reminded that there were no right or wrong answers and that the VA activity was merely a reflection exercise. VA students were instructed to select the two or three values that were most important to them from a list of 12 values. The values provided were identical to those used in previous VA studies and consisted of: being good at art; creativity; relationships with family and friends; government or politics; independence; learning and gaining knowledge; athletic ability; belonging to a social group (such as your community, racial group, or school club); music; career; spiritual or religious values; and sense of humour. After selecting values, VA students were prompted to spend 10 min writing about why these values are important to them. Control-group participants were only asked to complete the survey. In Week 7 of the semester, this procedure was repeated. Stereotype threat and social belonging items were included at Time 1 along with measures of internal and external motivation, commitment to values, and psychological flexibility. At Time 2, measures of social belonging, commitment to values, motivation, psychological flexibility, and school engagement were included. These measures were included to explore possible mediating variables between the intervention and GPA and to mask our interest in stereotype threat and social belonging to participants (no mediational analyses are included in the current study).

Measures

Student ID numbers from consenting participants were used to obtain semester GPAs and the following demographic variables: participants’ racial/ethnic backgrounds, first-generation status, and standardised Qscore – a value that is a composite of high school GPA and scores on college entrance exams.

Stereotype threat

At baseline, students completed a three-item adapted version of the original Stereotype Vulnerability Scale (SVS; Spencer, 1994; Woodcock, Hernandez, Estrada, & Schultz, 2012). The original SVS was used to capture the impact of stereotype threat on maths performance among women, but the scale has been modified to capture stereotypes related to ethnicity (Woodcock et al., 2012). The scale is psychometrically valid and has good reliability (Woodcock et al., 2012; α = .92 in the current study). Students responded to items on a 5-point scale (1 = never; 5 = almost always) that assessed how frequently they feel performance is attributed to ethnicity: “In school, I worry that people will draw conclusions about my ethnic group, based on the performances of other people in my ethnic group”; “In school, I worry that people will draw conclusions about my ethnic group based on my performances”; and “In school, I worry that people will draw conclusions about me, based on what they think about my ethnic group”.

Social belonging

We selected a short social belonging measure conceptualised to be particularly relevant for college students (Hurtado & Carter, 1997). At baseline, students completed a three-item measure of social belonging originally developed by Hurtado and Carter (1997) to assess Latino college students’ sense of belonging in the broader campus environment when starting school: “I see myself as a part of the university community”; “I feel that I am a member of the university community”; and “I feel a sense of belonging to the university community”. Responses were given on a 7-point scale (1 = very strongly disagree; 7 = very strongly agree), and the scale has good reliability (Hurtado & Carter, 1997; α = .93 in the current study).

Results

Missing data and preliminary results

As an initial step, we examined missing data on all variables of interest. Rates of missingness at Time 1 ranged from 9.0% to 13.0%, with the highest rates of missingness on the three stereotype threat items (all 13%). Missingness was not significantly related to any measured variable in the study. To account for missing data, we used multiple imputation (MI) with 10 iterations with all measured variables included in the imputation model. MI is seen as preferable over single imputation of missing data as missing values are estimated using available data from participants (Schafer & Graham, 2002).

To ensure conditions were statistically equivalent at baseline, we conducted the Pearson χ2 test of independence comparing gender, URM, and first-generation status and one-way analyses of variance (ANOVAs) comparing social belonging, stereotype vulnerability, and Qscore across conditions. Results revealed no significant baseline differences between control and treatment. Table 2 includes descriptive information for both control and treatment conditions, as well as Pearson χ2 and F statistics to test the equivalence of means. Bivariate correlations were also conducted to examine the relationships between variables of interest (Table 3).

Table 2.

Descriptive statistics by condition.

Variable Whole Sample
(N = 496)		 Control
(n = 107)		 Treatment
(n = 389)		 Pearson χ2 and F (df)
Gender 70% women;
30% men		 71% women;
29% men		 69% women;
31% men		 .15
URM students 92% 91% 93% .30
First-generation students 56% 54% 56% .22
URM and first-generation 50% 49% 52% .05
Social belonging
(1–7 scale)		 M = 4.94; SD = 1.15 M = 5.14; SD = 1.17 M = 4.88; SD = 1.14 2.52 (269)
Stereotype vulnerability
(1–5 scale)		 M = 2.71; SD = 1.25 M = 2.64; SD = 1.25 M = 2.73; SD = 1.25 .22 (259)
Standardised Qscore M = .04; SD = .97 M = .16; SD = .85 M = .03; SD = 1.00 2.11 (454)
Open in a new tab

Note: Pearson χ2 was used for nominal variables (gender, URM, first-generation, and URM + first-generation). ANOVA was used to test differences on continuous variables (social belonging, stereotype vulnerability, and Qscore). There were no significant differences between treatment and control.

Table 3.

Bivariate correlations between variables.

Social Belonging SVS Gender URM First-Generation QScore GPA
Social belonging –
SVS −.15* –
Gender −.07 .01 –
URM −.10 .18** −.12* –
First-generation .14* .03 .07 −.04 –
Qscore −.06 .19** −.08 .36*** −.27*** –
GPA −.05 .12 .04 .21*** −.08 .34*** –
Open in a new tab

Note: SVS = stereotype vulnerability; gender (0 = men, 1 = women); URM (0 = no, 1 = yes); first-generation (0 = no, 1 = yes).

***

p < .001.

**

p < .01.

*

p < .05.

Student engagement

Given the importance of participant engagement in successful interventions (Berkel, Mauricio, Schoenfelder, & Sandler, 2011) and that online interventions often lead to lower levels of engagement (Kohl, Crutzen, & de Vries, 2013), we examined engagement in an exploratory manner. We measured engagement in two ways: (a) length of students’ written responses to the values prompt and (b) number of times the intervention was completed (dosage).

VA participants were asked to complete the intervention twice and were instructed to spend 10 min writing about why their values were important to them; however, only 49 participants selected their top values twice (12.6% of the treatment group), and only 43 responded to the writing prompt at both time points (11.1% of the treatment group; all participants who responded to the writing prompt also selected their top values). At Time 1, of the VA participants who completed survey items 78.7% selected their top values and 70.7% responded to the writing prompt. For those who did write at Time 1, the mean response length was 118.33 words (SD = 74.77, Median = 104). For context, the following response was the median response length:

Having good relationships with family and friends are important to me because they are the people I would surround myself with to gain memories that will last. Independence is important to me because I cannot always rely on people for help, even with the smallest things. To learn things in life, I believe that sometimes we just have to figure them out for ourselves. I also chose athletic ability as a value that is important to me because I believe being confident in myself, I must be somewhat capable of being athletic because being up and moving puts me in a better mood.

At Time 2, fewer treatment group participants completed the writing portion of the intervention (Table 1; 76.0% selected their values and 67.2% provided a response to the writing prompt), and the average response was shorter (M = 102.95 words, SD = 71.08, Median = 92 words). In the treatment group, there were no significant differences on any baseline variable (e.g., gender, Qscore) between participants who completed survey measures but did not select values or complete the writing reflection, compared to participants who selected values and completed the reflection. Similarly, participants who selected their values but did not complete the reflection did not differ significantly on any variable from treatment participants who only completed survey items or participants who both selected values and completed the reflection. Similarly, there were no significant differences between participants who completed the intervention only at Time 1 compared to those who completed the intervention only at Time 2.

Direct and moderating effects

We tested our hypotheses through multiple hierarchical regression and utilised intent-to-treat analysis (i.e., VA participants who did not complete the intervention were still included in the treatment group). Intent-to-treat analysis can lead to underestimated programme effects but is the recommended approach when assessing the effectiveness of an RCT as it is more representative of the impact of an intervention outside of a research setting (Flay et al., 2005). First, GPA was regressed on condition (0 = control, 1 = treatment) with only URM students (Hypothesis 1a); second, GPA was regressed on condition with only first-generation students (Hypothesis 1b). In both regression analyses, gender (0 = men, 1 = women) and Qscore were included as covariates, as both have been shown to predict GPA (Westrick, Le, Robbins, Radunzel, & Schmidt, 2015). When examining direct effects for URM students, first-generation status was included as a covariate, and when testing direct effects for first-generation students, URM status was included as a covariate (Blocks 1 in Table 4). The effect of the VA was significant in predicting semester GPA for URM students; however, the effect was in the opposite direction from what was anticipated. URM students in the VA group had significantly lower GPAs than those in the control group. Additionally, the treatment regression coefficient for first-generation students was nearly identical to the regression coefficient for URM students, but the effect was not statistically significant (p = .10), likely due to a decrease in statistical power as there were fewer first-generation students than URM students.

Table 4.

Summary of regression analysis for variables predicting semester GPA for URM and first-generation students.

Predictor Variable Block 1 Block 2
URM Students
Gender .09* .10*
First-Generation .01 .01
Standardised QScore .41*** .41***
Treatment −.09* −.08*
Stereotype Threat – .03
Social Belonging – .01
Stereotype Threat*Treatment – −.03
Social Belonging*Treatment – −.03
R2 .16 .16
ΔR2 .16*** .00
First-Generation Students
Gender .03 .04
URM .06 .07
Standardised QScore .24** .24**
Treatment −.10 −.08
Stereotype Threat – −.05
Social Belonging – .06
Stereotype Threat*Treatment – −.03
Social Belonging*Treatment – −.10
R2 .16 .18
ΔR2 .16*** .02
Open in a new tab

Note: Gender (0 = men; 1 = women); first-generation (0 = no; 1 = yes); URM (0 = no; 1 = yes).

***

p < .001.

**

p < .01.

*

p < .05.

Moderating hypotheses were tested separately for URM and first-generation students through multiple regressions testing the effects of treatment, baseline stereotype threat, baseline social belonging, and the treatment by stereotype threat and treatment by social belonging interactions included in the models while controlling for gender, first-generation status (for URM students), URM status (for first-generation students), and standardised Qscore (Blocks 2 in Table 4). Both the stereotype threat by treatment and social belonging by treatment interactions were non-significant for both URM and first-generation students, indicating that the VA had a similar impact on students regardless of their baseline levels of stereotype threat and social belonging.

Post-hoc analyses

Given the unexpected negative findings, we ran post-hoc analyses in an attempt to gain more clarity on why the VA was ineffective or perhaps even iatrogenic in this sample. Because past research has demonstrated that the VA is less effective in schools that are less threatening for URM students (Hanselman et al., 2014), we conducted a t test to compare URM and non-URM students on stereotype threat. Our rationale was that if these two groups were comparable on stereotype threat, this particular college environment may be perceived as less threatening to URM students than other environments; however, results showed a significant difference between URM and non-URM students on stereotype threat (URM M = 2.87; non-URM M = 2.11; t = 4.66; p < .001).

We then assessed if participant engagement impacted the effectiveness of the intervention in multiple ways. First, we tested the impact of writing and the writing by stereotype threat and writing by social belonging interactions on semester GPA through multiple regression, again controlling for relevant covariates. Responses to the writing prompts were not related to semester GPA regardless of if the variable was calculated as dichotomous (did participant write “no” or “yes”; 0 = no, 1 = yes) or continuous (how much participant wrote). This was true when comparing (a) treatment participants who wrote versus control-group participants, (b) treatment participants who wrote versus those in the treatment group who did not, and (c) treatment participants who wrote versus those who did not (combined control group and VA group non-writers; Table 5 presents results comparing those who wrote in the treatment group to the control group). The null effect of response length is consistent with the study by Hanselman and colleagues (2017), which found that the length of written response was comparable between both effective and ineffective VA interventions. We also tested the impact of dosage (i.e., the number of times a participant completed the treatment; 0, 1, 2) and the stereotype threat by dosage and social belonging by dosage interactions through multiple regression controlling for relevant covariates (Table 6). Participants were considered to have received a dose of the VA if they at least selected their personal values from the list. Dosage was unrelated to semester GPA, and the dosage by stereotype threat and the dosage by social belonging interactions were also not significant. This was true when comparing those who received a dose of the VA to the control group as well as when comparisons were made within the treatment group (Table 6 presents results comparing participants in the treatment group who received a dose of the VA to the control group).

Table 5.

Summary of regression analysis for effect of writing on GPA.

Did Participant Write at either Time Point? Total Word Count
Variable B (SE) (95% CI) β B (SE) (95% CI) β
Stereotype Threat .04 (.06) (−.07, .15) .05 .03 (.06) (−.08, .14) .03
Social Belonging −.02 (.06) (−.13, .08) −.03 −.01 (.05) (−.11, .10) −.01
Response to Writing Prompt −.07 (.06) (−.12, .01) −.06 −.00 (.05) (−.05, .05) .00
Stereotype Threat*Writing −.06 (.06) (−.17, .05) −.08 .07 (.05) (−.03, .17) .08
Social Belonging*Writing .07 (.06) (−.04, .17) .08 .04 (.05) (−.06, .14) .05
R2 .17*** .19***
Open in a new tab

Note: Gender, first-generation status, URM, and Qscore were included as covariates; 0 = participant did not write, 1 = participant did write.

***

p < .001.

**

p < .01.

*

p < .05.

Table 6.

Summary of regression analysis for effect of dosage on GPA.

Variable B (SE) (95% CI) β
Stereotype Threat −.04 (.10) (−.23, .15) −.04
Social Belonging .04 (.14) (−.24, .32) .03
Dose −.13 (.11) (−.34, .08) −.07
Stereotype Threat*Dose .02 (.05) (−.07, .11) .04
Social Belonging*Dose .09 (.15) (−.21, .39) .08
R2 .18***
Open in a new tab

Note: Dose coded 0, 1, 2. Participants were considered receiving a dose of the VA if they selected personal values. Gender, first-generation status, URM, and Qscore were included as covariates.

***

p < .001.

**

p < .01.

*

p < .05.

Discussion

Past research has demonstrated the utility of the VA for academic performance among both URM students (Cohen et al., 2006; Cohen, Garcia, Purdie-Vaughns, Apfel, & Brzustoski, 2009; Sherman et al., 2013) and first-generation students (Harackiewicz et al., 2014); however, this was not the case in the current study. URM students in the VA group had lower GPAs than those in the control group, and the same was true among first-generation students although the effect was not statistically significant. Additionally, previous studies have shown the VA to be an effective tool in buffering against stereotype threat (Miyake et al., 2010) and supporting students with low levels of social belonging (Cook, Purdie-Vaughns, Garcia, & Cohen, 2011; Harackiewicz et al., 2014); again, the results from our study are inconsistent with these findings, as neither baseline stereotype threat nor social belonging moderated the impact of the intervention.

Iatrogenic effect

To our knowledge, this is the first VA study with results suggesting that participating in a values affirmation intervention may produce an iatrogenic effect. Although there is little evidence to suggest that brief social-psychological interventions can have unintended negative effects, particularly in terms of academic achievement, Long, Renshaw, and Camarota (2018) found that a mindfulness-based intervention had a negative effect on students’ subjective wellbeing – a composite measure of joy of learning, school connection, educational purpose, and academic efficacy. Although mindfulness is unique from values reflection, Acceptance and Commitment Therapy (ACT; Hayes, 2004) proposes that mindfulness and awareness of values are important, inter-related aspects of ACT that influence one another. If the VA negatively impacted the same construct as in the Long and colleagues’ (2018) study, then it could have led to lower student GPAs through decreases in subjective wellbeing, as GPA and wellbeing have been previously linked (Renshaw et al., 2014).

An alternative explanation lies in the VA intervention itself. Having new college students reflect on their values could potentially be problematic if they felt they were unable to direct their lives in a way that aligns with their values (Hayes, Luoma, Bond, Masuda, & Lillis, 2006). For example, if a student valued belonging, but felt isolated in the university environment, bringing these thoughts of belonging to the forefront of consciousness could have had an unintended negative effect on the student, as they felt a disconnect between their values and their experience. This explanation is speculative, and future studies are certainly needed to further unpack possible iatrogenic effects of the VA.

Why was the VA ineffective?

Regardless of the interpretation of the iatrogenic effect, it is clear that the VA was not effective for URM and first-generation students, at least in terms of their academic achievement. In the current study, the VA was delivered electronically, which led to multiple alterations that may have impacted the effectiveness of the VA. Past VA interventions have typically been delivered as a class activity or assignment, which carries a unique incentive to complete (e.g., Cohen et al., 2006; Miyake et al., 2010). While an incentive was provided in the current study in the form of entry to a gift-card lottery, this presents a dramatically different motivator than an assignment. Additionally, providing materials online, removed from a specific course, may have influenced the effectiveness of the intervention as well. Research has shown that people are more likely to experience attention lapses and have lower levels of recall when materials are presented online as opposed to in person (Risko, Anderson, Sarwal, Engelhardt, & Kingstone, 2012). This is meaningful for a couple of reasons. If students are not engaging and focusing on the values intervention to the same degree as students do in a class, then the intervention is likely to have a diluted effect. Additionally, it is possible that classroom-based VA interventions are effective in part because students’ memories of the VA are triggered when they later return to the same classroom; students in the current study may have been less likely to revisit their values reflection after completion than if they had completed the intervention as a part of a class. Delivering the intervention online, without a connection to a specific course, essentially may have provided a more distal version of the intervention that was less likely to be recalled when it might have been useful (e.g., during an exam).

Previous studies have provided evidence that the VA is only effective in the presence of a stressor. In this study, participants likely completed the intervention in a variety of settings (e.g., at their residence hall or at home). It is possible that for students completing the intervention outside of a classroom setting, without thinking about academic assessments, stereotype threat is not activated to the same extent as when they are around their peers in an academic setting. Similarly, concerns about social belonging may not be as salient once students are outside of the classroom. Feelings of belonging are largely dependent on our perception of the environment around us; when we perceive an environment as threatening, we question if we belong in that environment (Crocker, Luhtanen, & Sommers, 2004). Removing the classroom and assignment components from the VA intervention and allowing students to complete the intervention in whichever environment they chose may have reduced stereotype and social belonging threats, rendering the VA less necessary. Additionally, the survey and VA were delivered electronically through MSS, an organisation that participants may have had a level of comfort and familiarity with given MSS’s goal of supporting underrepresented students. While partnering with MSS likely increased the number of students we were able to reach in this study, it is also possible that having the VA delivered from a source perceived by students as supportive ultimately reduced participants’ concerns regarding potential negative stereotypes and a lack of belonging at the university. Essentially, allowing students to complete the VA outside of a classroom setting may have prevented the activation of stereotype and belonging threats, and delivering the VA through MSS may have buffered against the presence of these threats, both of which would have likely negated the importance of the intervention.

Finally, it is possible that too few participants completed the VA as intended. Prior studies assessing GPAs as an outcome have typically delivered the VA twice with both the selection of values and written reflection occurring at both time points. However, in the current study only 49 participants selected their personal values twice and only 43 completed the written reflection at both time points. Additionally, participants who did respond to the written prompts wrote on average 111.3 words. While there is a dearth of literature on the length of written VA responses, the average response length is markedly different from the reported average response in the study by Harackiewicz and colleagues (2014; response length M = 135.39; Tibbetts et al., 2016). Furthermore, our response rate of 29% is dramatically different from other studies that recruited and delivered the intervention through a course. For example, 99% of eligible participants participated in the study by Harackiewicz and colleagues (2014), and 73% of students in the study by Miyake and colleagues (2010) completed both writing exercises. While we tested the impact of engagement in multiple ways, all of which were unrelated to GPA, our unanticipated findings may be the result of a sample size that is too small or not representative.

Limitations

Our study is not without limitations. A pivotal piece of the VA is when it is delivered, as the intervention should be implemented when there is some sort of adversity (Cohen & Sherman, 2014). As such, when the outcome of interest is academic performance, the VA intervention is delivered during the first or second week of the semester, and right before midterm exams; each is a time of additional stress and key time points for academic success (Bland et al., 2012; DeBerard, Spielmans, & Julka, 2004). While we tried to replicate the timing of the delivery, the VA was available for 2 weeks to provide students ample opportunity to complete the intervention and increase the study’s response rate. As such, there was substantial variability in when participants completed the intervention. Additionally, these students had different class schedules, so it was not possible to ensure each student completed the intervention in the week before a midterm exam, as there is variability across classes in when (and if) exams are delivered. While it is likely that students who completed the VA did so at a period of heightened stress (beginning of the semester and around midterms), there is also the possibility that some students waited until a time where their stress had declined before completing the intervention (e.g., completed midterm exams and then completed the survey and VA).

Engagement with the VA was unrelated to semester GPA. While the non-significant findings should be interpreted with caution due to the low number of students who completed the intervention twice and wrote about their values, which inflated the Type II error rate (Freiman, Chalmers, Smith, & Kuebler, 1978), it is important to note that URM students in the VA condition had significantly lower GPAs than URM students in the control group. Similarly, while it was not a statistically significant difference, first-generation students in the VA group had lower GPAs than the control group.

We attempted to examine the role engagement played in the effectiveness of the VA by looking at both direct and moderational effects of the length of response to the VA writing prompts. However, word count may not be a perfect measure of engagement, as it is possible that some students spent the instructed amount of time on the prompt, but this was not reflected in the length of their responses. For example, students were able to complete the VA on their phone, which could have impacted the length of their responses, and some students may write more slowly than others.

While we assume the level of stereotype and social belonging participants encountered was different when completing the intervention outside of a classroom, we cannot say this definitively, as we had no record of where participants completed the intervention and we did not ask participants to track their stereotype vulnerability and feelings of belonging in multiple settings. Finally, there was a potential bias due to non-response error; many students participated in the values affirmation but did not provide all information at each time point. While treatment participants who completed the VA twice did not differ significantly from participants who only completed the VA once, or from participants who provided information in the survey and did not complete the VA, they may be different from the general population of URM and first-generation students on critical unmeasured characteristics (e.g., conscientiousness), which ultimately made the VA less valuable for them.

Future directions

We are left to speculate on the apparent iatrogenic effect for URM students; it will be important to devote further attention to this issue in future studies. Continuing to examine whether the VA is more suitable in some contexts than others (e.g., students new to college) will be crucial. Additionally, it is important to continue to unpack the impact engagement and setting have on the effectiveness of the VA. Common VA instructions direct participants to reflect on and write about their values for 10 to 15 min (e.g., Harackiewicz et al., 2014); however, we are unaware of any study that has identified any sort of threshold that is required for the intervention to be successful. This could be tested through randomly modifying the length of the values prompt (e.g., 5 min vs. 10 min vs. 15 min). To tease apart the impact of setting, participants could be randomly assigned to complete the intervention in the classroom or at home with the control group completing the intervention in-class or at home. This would allow researchers to make comparisons between the two treatment groups to assess if the removal of the classroom environment impacted the effectiveness of the VA. Additionally, while we believe that stereotype threat and social belonging are likely impacting participants differently when the intervention is no longer tied to an assignment and the classroom setting is removed, it is important that this is further assessed.

Conclusion

In summation, this study adds to the values affirmation literature in multiple ways. As indicated by the high percentage of participants who did not write about why their values were important to them or who only completed the intervention once, students will not engage with the electronic intervention protocol to the same extent as when it is delivered as a class assignment; additional incentives outside of a gift card lottery appear to be necessary. To our knowledge, this is the first study to report iatrogenic effects of the intervention. The potential iatrogenic effect for students further indicates that the context the VA is delivered in matters. While increasing the reach of the VA intervention may be desirable, attention must be paid to replication.

Acknowledgement

The project described was supported by awards P50 DA010075 and T32 DA017629 from the National Institute on Drug Abuse. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institute on Drug Abuse or the National Institutes of Health.

Footnotes

Disclosure statement

No potential conflict of interest was reported by the authors.

References

  1. Bratter JL, Rowley KJ, & Chukhray I (2016). Does a self-affirmation intervention reduce stereotype threat in black and Hispanic high schools? Race and Social Problems, 8(4), 340–356. doi: 10.1007/s12552-016-9187-4 [DOI] [Google Scholar]
  2. Berkel C, Mauricio AM, Schoenfelder E, & Sandler IN (2011). Putting the pieces together: An integrated model of program implementation. Prevention Science, 12(1), 23–33. doi: 10.1007/s11121-010-0186-1 [DOI] [PMC free article] [PubMed] [Google Scholar]
  3. Bland HW, Melton BF, Welle P, & Bingham L (2012). Stress tolerance: New challenges for millennial college students. College Student Journal, 46(2), 362–275. [Google Scholar]
  4. Cohen GL, Aronson J, & Steele CM (2000). When beliefs yield to evidence: Reducing biased evaluation by affirming the self. Personality and Social Psychology Bulletin, 26(9), 1151–1164. doi: 10.1177/01461672002611011 [DOI] [Google Scholar]
  5. Cohen GL, Garcia J, Apfel N, & Master A (2006). Reducing the racial achievement gap: A social-psychological intervention. Science, 313(5791), 1307–1310. doi: 10.1126/science.1128317 [DOI] [PubMed] [Google Scholar]
  6. Cohen GL, Garcia J, Purdie-Vaughns V, Apfel N, & Brzustoski P (2009). Recursive processes in self-affirmation: Intervening to close the minority achievement gap. Science, 324(5925), 400–403. doi: 10.1126/science.1170769 [DOI] [PubMed] [Google Scholar]
  7. Cohen GL, & Sherman DK (2014). The psychology of change: Self-affirmation and social psychological intervention. Annual Review of Psychology, 65, 333–371. doi: 10.1146/annurev-psych-010213-115137 [DOI] [PubMed] [Google Scholar]
  8. Cook J, Purdie-Vaughns V, Garcia J, & Cohen G (2011, Mar-Apr). Longitudinal impact of a values affirmation and its timing on minority students’ belonging and performance in middle school. Paper presented at the Biennial Meeting for the Society for Research in Child Development, Montreal, Canada. [Google Scholar]
  9. Creswell JD, Welch WT, Taylor SE, Sherman DK, Gruenewald TL, & Mann T (2005). Affirmation of personal values buffers neuroendocrine and psychological stress responses. Psychological Science, 16(11), 846–851. doi: 10.1111/j.1467-9280.2005.01624.x [DOI] [PubMed] [Google Scholar]
  10. Crocker J, Luhtanen RK, & Sommers SR (2004). Contingencies of self-worth: Progress and prospects. European Review of Social Psychology, 15(1), 133–181. doi: 10.1080/10463280440000017 [DOI] [Google Scholar]
  11. DeBerard MS, Spielmans GI, & Julka DL (2004). Predictors of academic achievement and retention among college freshmen: A longitudinal study. College Student Journal, 38(1), 66–80. [Google Scholar]
  12. Dee TS (2015). Social identity and achievement gaps: Evidence from an affirmation intervention. Journal of Research on Educational Effectiveness, 8(2), 149–168. doi: 10.1080/19345747.2014.906009 [DOI] [Google Scholar]
  13. Fein S, & Spencer SJ (1997). Prejudice as self-image maintenance: Affirming the self through derogating others. Journal of Personality and Social Psychology, 73(1), 31–44. doi: 10.1037/0022-3514.73.1.31 [DOI] [Google Scholar]
  14. Flay BR, Biglan A, Boruch RF, Castro FG, Gottfredson D, Kellam S, … Ji P (2005). Standards of evidence: Criteria for efficacy, effectiveness and dissemination. Prevention Science, 6(3), 151–175. doi: 10.1007/s11121-005-5553-y [DOI] [PubMed] [Google Scholar]
  15. Freiman JA, Chalmers TC, Smith H Jr., & Kuebler RR (1978). The importance of beta, the type II error and sample size in the design and interpretation of the randomized control trial: Survey of 71 negative trials. New England Journal of Medicine, 299(13), 690–694. doi: 10.1056/NEJM197809282991304 [DOI] [PubMed] [Google Scholar]
  16. Griffiths F, Lindenmeyer A, Powell J, Lowe P, & Thorogood M (2006). Why are health care interventions delivered over the internet? A systematic review of the published literature. Journal of Medical Internet Research, 8(2), e10. doi: 10.2196/jmir.8.2.e10 [DOI] [PMC free article] [PubMed] [Google Scholar]
  17. Hanselman P, Bruch SK, Gamoran A, & Borman GD (2014). Threat in context: School moderation of the impact of social identity threat on racial/ethnic achievement gaps. Sociology of Education, 87(2), 106–124. doi: 10.1177/0038040714525970 [DOI] [Google Scholar]
  18. Hanselman P, Rozek CS, Grigg J, & Borman GD (2017). New evidence on self-affirmation effects and theorized sources of heterogeneity from large-scale replications. Journal of Educational Psychology, 109(3), 405–424. doi: 10.1037/edu0000141 [DOI] [PMC free article] [PubMed] [Google Scholar]
  19. Harackiewicz JM, Canning EA, Tibbetts Y, Giffen CJ, Blair SS, Rouse DI, & Hyde JS (2014). Closing the social class achievement gap for first-generation students in undergraduate biology. Journal of Educational Psychology, 106(2), 375–389. doi: 10.1037/a0034679 [DOI] [PMC free article] [PubMed] [Google Scholar]
  20. Harackiewicz JM, Canning EA, Tibbetts Y, Priniski SJ, & Hyde JS (2016). Closing achievement gaps with a utility-value intervention: Disentangling race and social class. Journal of Personality and Social Psychology, 111(5), 745–765. doi: 10.1037/pspp0000075 [DOI] [PMC free article] [PubMed] [Google Scholar]
  21. Hayes SC (2004). Acceptance and commitment therapy, relational frame theory, and the third wave of behavioral and cognitive therapies. Behavior Therapy, 35(4), 639–665. doi: 10.1016/S0005-7894(04)80013-3 [DOI] [PubMed] [Google Scholar]
  22. Hayes SC, Luoma JB, Bond FW, Masuda A, & Lillis J (2006). Acceptance and commitment therapy: Model, processes and outcomes. Behaviour Research and Therapy, 44(1), 1–25. doi: 10.1016/j.brat.2005.06.006 [DOI] [PubMed] [Google Scholar]
  23. Hurtado S, & Carter DF (1997). Effects of college transition and perceptions of the campus racial climate on Latino college students’ sense of belonging. Sociology of Education, 70(4), 324–345. doi: 10.2307/2673270 [DOI] [Google Scholar]
  24. Jordt H, Eddy SL, Brazil R, Lau I, Mann C, Brownell SE, … Freeman S (2017). Values affirmation intervention reduces achievement gap between underrepresented minority and white students in introductory biology classes. CBE-Life Sciences Education, 16(3), ar41. doi: 10.1187/cbe.16-12-0351 [DOI] [PMC free article] [PubMed] [Google Scholar]
  25. Kizilcec RF, Saltarelli AJ, Reich J, & Cohen GL (2017). Closing global achievement gaps in MOOCs. Science, 355(6322), 251–252. doi: 10.1126/science.aag2063 [DOI] [PubMed] [Google Scholar]
  26. Kohl LFM, Crutzen R, & de Vries NK (2013). Online prevention aimed at lifestyle behaviors: A systematic review of reviews. Journal of Medical Internet Research, 15(7), e146. doi: 10.2196/jmir.2665 [DOI] [PMC free article] [PubMed] [Google Scholar]
  27. Kost-Smith LE, Pollock SJ, Finkelstein ND, Cohen GL, Ito TA, & Miyake A (2012). Replicating a self-affirmation intervention to address gender differences: Successes and challenges. AIP Conference Proceedings, 1413(1), 231–234. doi: 10.1063/1.3680037 [DOI] [Google Scholar]
  28. Layous K, Davis EM, Garcia J, Purdie-Vaughns V, Cook JE, & Cohen GL (2017). Feeling left out, but affirmed: Protecting against the negative effects of low belonging in college. Journal of Experimental Social Psychology, 69, 227–231. doi: 10.1016/j.jesp.2016.09.008 [DOI] [Google Scholar]
  29. Lazowski RA, & Hulleman CS (2016). Motivation interventions in education: A meta-analytic review. Review of Educational Research, 86(2), 602–640. doi: 10.3102/0034654315617832 [DOI] [Google Scholar]
  30. Long AC, Renshaw TL, & Camarota D (2018). Classroom management in an urban, alternative school: A comparison of mindfulness and behavioral approaches. Contemporary School Psychology, 22(3), 233–248. doi: 10.1007/s40688-018-0177-y [DOI] [Google Scholar]
  31. Martens A, Johns M, Greenberg J, & Schimel J (2006). Combating stereotype threat: The effect of self-affirmation on women’s intellectual performance. Journal of Experimental Social Psychology, 42(2), 236–243. doi: 10.1016/j.jesp.2005.04.010 [DOI] [Google Scholar]
  32. McQueen A, & Klein WMP (2006). Experimental manipulations of self-affirmation: A systematic review. Self and Identity, 5(4), 289–354. doi: 10.1080/15298860600805325 [DOI] [Google Scholar]
  33. Miyake A, Kost-Smith LE, Finkelstein ND, Pollock SJ, Cohen GL, & Ito TA (2010). Reducing the gender achievement gap in college science: A classroom study of values affirmation. Science, 330(6008), 1234–1237. doi: 10.1126/science.1195996 [DOI] [PubMed] [Google Scholar]
  34. Renshaw TL, Furlong MJ, Dowdy E, Rebelez J, Smith DC, O’Malley MD, … Strøm IF (2014). Covitality: A synergistic conception of adolescents’ mental health In Furlong MJ, Gilman R, & Huebner ES (Eds.), Handbook of positive psychology in the schools (2nd ed., pp. 12–32). New York, NY: Routledge. doi: 10.4324/9780203106525.ch2 [DOI] [Google Scholar]
  35. Risko EF, Anderson N, Sarwal A, Engelhardt M, & Kingstone A (2012). Everyday attention: Variation in mind wandering and memory in a lecture. Applied Cognitive Psychology, 26(2), 234–242. doi: 10.1002/acp.1814 [DOI] [Google Scholar]
  36. Schafer JL, & Graham JW (2002). Missing data: Our view of the state of the art. Psychological Methods, 7(2), 147–177. doi: 10.1037/1082-989X.7.2.147 [DOI] [PubMed] [Google Scholar]
  37. Sherman DK, & Cohen GL(2006). The psychology of self-defense: Self-affirmation theory In Zanna MP (Ed.), Advances in experimental social psychology (Vol. 38, pp. 183–242). San Diego, CA: Academic Press; doi: 10.1016/S0065-2601(06)38004-5 [DOI] [Google Scholar]
  38. Sherman DK, Hartson KA, Binning K, Purdie-Vaughns V, Garcia J, Taborsky-Barba S, … Cohen GL (2013). Deflecting the trajectory and changing the narrative: How self-affirmation affects academic performance and motivation under identity threat. Journal of Personality and Social Psychology, 104(4), 591–618. doi: 10.1037/a0031495 [DOI] [PubMed] [Google Scholar]
  39. Sherman DAK, Nelson LD, & Steele CM (2000). Do messages about health risks threaten the self? Increasing the acceptance of threatening health messages via self-affirmation. Personality and Social Psychology Bulletin, 26(9), 1046–1058. doi: 10.1177/01461672002611003 [DOI] [Google Scholar]
  40. Shnabel N, Purdie-Vaughns V, Cook JE, Garcia J, & Cohen GL (2013). Demystifying values-affirmation interventions: Writing about social belonging is a key to buffering against identity threat. Personality and Social Psychology Bulletin, 39(5), 663–676. doi: 10.1177/0146167213480816 [DOI] [PubMed] [Google Scholar]
  41. Spencer SJ (1994). The effect of stereotype vulnerability on women’s math performance. Dissertation Abstracts International: Section B. The Sciences and Engineering, 54(7-B), 3903. [Google Scholar]
  42. Steele CM, & Aronson J (1995). Stereotype threat and the intellectual test performance of African Americans. Journal of Personality and Social Psychology, 69(5), 797–811. doi: 10.1037/0022-3514.69.5.797 [DOI] [PubMed] [Google Scholar]
  43. Steele CM, & Liu TJ (1983). Dissonance processes as self-affirmation. Journal of Personality and Social Psychology, 45(1), 5–19. doi: 10.1037/0022-3514.45.1.5 [DOI] [Google Scholar]
  44. Stephens NM, Townsend SSM, Markus HR, & Phillips LT (2012). A cultural mismatch: Independent cultural norms produce greater increases in cortisol and more negative emotions among first-generation college students. Journal of Experimental Social Psychology, 48(6), 1389–1393. doi: 10.1016/j.jesp.2012.07.008 [DOI] [Google Scholar]
  45. Tibbetts Y, Harackiewicz JM, Canning EA, Boston JS, Priniski SJ, & Hyde JS (2016). Affirming independence: Exploring mechanisms underlying a values affirmation intervention for first-generation students. Journal of Personality and Social Psychology, 110(5), 635–659. doi: 10.1037/pspa0000049 [DOI] [PMC free article] [PubMed] [Google Scholar]
  46. Walton GM, & Carr PB (2012). Social belonging and the motivation and intellectual achievement of negatively stereotyped students In Inzlicht M & Schmader T (Eds.), Stereotype threat: Theory, process, and application (pp. 89–106). New York, NY: Oxford University Press. [Google Scholar]
  47. Walton GM, & Cohen GL (2011). A brief social-belonging intervention improves academic and health outcomes of minority students. Science, 331(6023), 1447–1451. doi: 10.1126/science.1198364 [DOI] [PubMed] [Google Scholar]
  48. Westrick PA, Le H, Robbins SB, Radunzel JM, & Schmidt FL (2015). College performance and retention: A meta-analysis of the predictive validities of ACT® scores, high school grades, and SES. Educational Assessment, 20(1), 23–45. doi: 10.1080/10627197.2015.997614 [DOI] [Google Scholar]
  49. Woodcock A, Hernandez PR, Estrada M, & Schultz P (2012). The consequences of chronic stereotype threat: Domain disidentification and abandonment. Journal of Personality and Social Psychology, 103(4), 635–646. doi: 10.1037/a0029120 [DOI] [PMC free article] [PubMed] [Google Scholar]
  50. Yeager DS, & Walton GM (2011). Social-psychological interventions in education: They’re not magic. Review of Educational Research, 81(2), 267–301. doi: 10.3102/0034654311405999 [DOI] [Google Scholar]

RESOURCES