Abstract
First-generation college students (students for whom neither parent has a 4-year college degree) earn lower grades and worry more about whether they belong in college, compared to continuing-generation students (who have at least one parent with a 4-year college degree). We conducted a longitudinal follow-up of participants from a study in which a values-affirmation intervention improved performance in a biology course for first-generation college students, and found that the treatment effect on grades persisted three years later. First-generation students in the treatment condition obtained a GPA that was, on average, .18 points higher than first-generation students in the control condition, three years after values affirmation was implemented (Study 1A). We explored mechanisms by testing if the values-affirmation effects were predicated on first-generation students reflecting on interdependent values (thus affirming their values that are consistent with working-class culture) or independent values (thus affirming their values that are consistent with the culture of higher education). We found that when first-generation students wrote about their independence, they obtained higher grades (both in the semester in which values affirmation was implemented and in subsequent semesters) and felt less concerned about their background. In a separate laboratory experiment (Study 2) we manipulated the extent to which participants wrote about independence and found that encouraging first-generation students to write more about their independence improved their performance on a math test. These studies highlight the potential of having FG students focus on their own independence.
Keywords: social class, achievement gap, first-generation students, values affirmation
First-generation (FG) college students – for whom neither parent has a four-year degree – comprise over 20% of enrolled college students (Chen, 2005), but they face a number of economic and social barriers and tend to struggle in their college courses. Compared to continuing-generation (CG) students – for whom at least one parent has a four-year degree – FG students perform more poorly in college, have higher drop-out rates, and report more difficulty adapting to college (Sirin, 2005; Terenzini, Springer, Yaeger, Pascarella, & Nora, 1996). The higher dropout rate carries unfortunate economic consequences, as college graduates earn over 95% more in their weekly salaries than individuals with only a high school diploma (Autor, 2014). Researchers consider parental education to be a proxy for social class, and thus the performance gap between FG and CG students has been referred to as the social-class achievement gap (Jackman & Jackman, 1983; Pascarella & Terenzini, 1991; Snibbe & Markus, 2005; Stephens, Fryberg, Markus, Johnson, & Covarrubias, 2012). This social-class achievement gap is due, in part, to the lack of resources available to FG students. In addition to having less parental guidance for navigating higher education, FG students are more likely to come from working class backgrounds or poverty (Reardon, 2011; Saenz, Hurtado, Barrera, Wolf, & Yeung, 2007), and attend lower quality high schools that may lack the academic rigor required for students to excel in college (Terenzini et al., 1996; Warburton, Bugarin, & Nunez, 2001).
The social-class achievement gap may also reflect psychological factors related to how FG students experience the college environment. Previous research indicates that FG students sometimes feel stereotyped and worry about “fitting in” at college, which carries detrimental consequences for their academic performance (Croizet & Claire, 1998; Harackiewicz et al., 2014; Johnson, Richeson, & Finkel, 2011; Ostrove & Long, 2007; Smeding, Darnon, Souchal, Toczek-Capelle, & Butera, 2013; Stephens, Fryberg, et al., 2012). In fact, one study demonstrated that the social class achievement gap in college GPAs was mediated by the lack of academic fit perceived by students from lower social class backgrounds (Ostrove & Long, 2007). That is, students from lower social class backgrounds reported feeling less like they belonged in college, and this difference accounted for their lower overall GPAs (Ostrove & Long, 2007). Many researchers have documented that FG students have concerns about academic fit (e.g., Harackiewicz et al., 2014; Johnson et al., 2011; Ostrove & Long, 2007) and a recent theory suggests that this could be due to FG students’ perceptions of university norms as being discrepant from their own motives for attending college (Stephens, Fryberg, et al., 2012). The research reported here addresses these social-class gaps in an introductory university biology course and has two components. The first is a longitudinal follow-up of students who participated in a values-affirmation intervention (or control), to determine whether the positive effects persisted 3 years later, and whether the effects involved reflection on interdependent or independent values. The second component is a laboratory experiment in which writing about independent values was experimentally manipulated.
Cultural Mismatch Theory
Stephens, Fryberg, et al. (2012) posited that FG students contend with identity threat due to a mismatch between their personal values and the institutional values implicit in university settings. In particular, they argued that FG students face an unseen disadvantage due to a cultural mismatch between the middle-class norms of independence reflected in the American university system and their own interdependent motives for attending college. Whereas a culture of independence may be familiar and comfortable to middle-class students, it can be experienced as threatening by many FG students who may have been socialized with more interdependent norms (i.e., being part of a community, and connecting with others). Indeed, research has demonstrated that FG students are more likely than CG students to report that interdependent motives, such as giving back to the community or helping their family after they finish college, are important reasons for completing their college degree (Harackiewicz et al., 2014; Stephens, Fryberg, et al., 2012). Thus, the independent focus of American universities (i.e., emphasis on personal development and achievement) may inadvertently function as a social identity threat to FG students, who tend to be more interdependent in motivational orientation (Stephens, Townsend, Markus, & Phillips, 2012).
Stephens, Fryberg, et al. (2012) found that university administrators were more likely to emphasize the importance of independent skills (e.g., working independently) than interdependent skills (e.g., working collaboratively in groups) and similarly, that CG freshmen endorsed more independent motives for attending college than interdependent motives. In contrast, FG students entered college with more interdependent and fewer independent motives for attending college than their CG peers. Furthermore, a match between motives for attending college and university norms (as was the case for CG students) versus a mismatch (as was the case for FG students) carried critical consequences for academic performance: independent motives for attending college were positively correlated with course grades over a two-year period, whereas interdependent motives for attending college were negatively related to grades. In fact, they found that the degree to which students endorsed independent and interdependent motives mediated the relationship between socio-economic status and academic performance during the first two years of college. Relatedly, Stephens, Townsend et al. (2012) have also demonstrated that FG students experience the independent culture of higher education as a more stressful and difficult environment than CG students.
One way to alleviate this cultural mismatch is to change students’ perceptions of the environment to be more consistent with their values (i.e., create a cultural match). In an experimental study, Stephens, Fryberg et al. (2012) varied the depiction of university norms in a welcome letter ostensibly written by the university president. They found that when the university culture was depicted as more interdependent (with an emphasis on working together, participating in collaborative research, and learning from others), FG students performed as well as CG students on subsequent achievement tasks. Conversely, when the culture was described as more independent (with an emphasis on creating your own intellectual journey and participating in independent research), FG students perceived the subsequent task as more difficult and performed more poorly than CG students.
These results suggest that changing the environment to create a cultural match between students’ motives for attending college and the academic environment can improve academic success. However, a different way to address the mismatch may be to help students focus on personal values that are consistent with the environment. In other words, when students’ values are experienced as inconsistent with the university context, there are two ways to address the mismatch – either by changing the perception of the context, or by helping students focus on their own values that are consistent with the educational context. Indeed, all college students, both CG and FG, typically endorse a combination of values, some independent, some interdependent, and the extent to which specific personal values become salient can be influenced by situational cues. We hypothesized that this mismatch could be alleviated when FG students reflect on their own independent values.
Values Affirmation
One intervention that has proven effective at leveraging students’ self-perceptions and values to improve academic performance is values affirmation. Values affirmation (VA) is predicated on affirming core personal values to establish a perception of self-integrity and self-worth that, in turn, alleviates stress and helps students address challenges (see McQueen & Klein, 2006; Cohen & Sherman, 2014, for reviews). When students are confronted with social identity threats, they experience anxiety that can impair performance on challenging academic tasks (Steele, 1997). By reflecting on their core values in a brief writing assignment, however, students can bolster their self-integrity, making identity threats less salient and enabling students to dedicate more cognitive resources to the relevant academic task (Cohen & Sherman, 2014). The VA technique has proven effective in promoting the academic performance of underrepresented groups who experience identity threat in evaluative settings. For example, VA interventions have successfully reduced racial achievement gaps in middle school (Cohen, Garcia, Apfel, & Master, 2006; Hanselman, Bruch, Gamoran, & Borman, 2014; Sherman et al., 2013), a gender gap in a college physics class (Miyake et al., 2010), and the social-class achievement gap in a college biology class (Harackiewicz, Canning et al. 2014).
Recent research suggests that VA effects may be driven by students’ tendencies to write about why their important values make them feel connected to other people (Shnabel, Purdie-Vaughns, Cook, Garcia, & Cohen, 2013). By writing about interpersonal connections and making students’ interdependence more salient, students are reminded of the social support available to them, and this may enable them to perform up to their full potential, even when contending with social identity threats. Shnabel et al. examined the content of essays from the original Cohen et al. (2006) study in which a VA intervention reduced the racial-achievement gap in middle school. Specifically, they coded the VA essays for themes of interdependence and found that Black seventh graders wrote significantly more about their social bonds and interpersonal connections in the affirmation condition (compared to the control condition). Mediation analyses revealed that writing about interpersonal connections with others mediated the positive effect of VA for Black students. Although Shnabel et al. referred to writing about interpersonal connections as affirming “social belonging,” we believe this is comparable to affirming interdependence and will use the term “interdependence” for consistency with the current analysis.
Shnabel et al. (2013) also manipulated the extent to which students wrote about interdependence in a laboratory study with White undergraduates. Some participants were encouraged to write about how their chosen values made them feel closer and more connected with others (i.e., affirm their interdependence) and some completed a standard VA exercise. They found that women performed better on a math test when they were encouraged to write about interpersonal connections in their VA essays. Considered together, these studies suggest that writing about interdependence in VA essays can be an effective way to promote academic performance for students facing identity threat. Shnabel et al. posited that writing about valued social bonds reminds threatened individuals of their meaningful connections with significant others which then bolsters their self-integrity and effectively buffers students from the negative consequences of identity threat. This could apply to FG students as well.
On the one hand, reflecting on interpersonal connections and interdependent values may provide emotional resources for FG students that help them cope with academic challenges. Conversely, focusing on interdependent values might exacerbate feelings of cultural mismatch, and could conceivably impair performance. Thus, writing about interdependence may or may not be a plausible mediator of VA effects for FG students. It also seems possible that reflecting on personal independence may help FG students feel more aligned with the independent culture of higher education, thereby fostering an increased sense of academic fit and better performance. It is important to note that reflecting on independence in VA essays need not come at the expense of affirming interdependence. For example, Shnabel et al. (2013) noted that many VA essays contained both themes, as students often wrote about both interpersonal connections and their independence in a single essay. Reflecting on their independence may in fact be beneficial for their academic performance and success in college.
Affirming independent values that are consistent with the college context and learning environment may benefit FG students by helping them feel more like they belong in college. Given that FG students’ predominant motivation for attending college is “mismatched” with university contexts that encourage independence and individuality, we hypothesize that it will be adaptive for FG students to reflect on the kinds of values promoted in higher education and write about why these values are personally important. In this way, affirming independence could foster a cultural match for FG students without compromising their interdependent motives.
Another line of support for this hypothesis comes from research on identity-based motivation, which has shown that when an activity or situation feels congruent with one’s identity, college students are more motivated, academically engaged, and perform better (Oyserman & Destin, 2010). A fundamental tenet of identity-based motivation is that identities are multi-faceted and context dependent. People interpret situations and difficulties differently depending on which identity or self-concept becomes activated, and they tend to prefer identity-congruent actions (Oyserman & Destin, 2010). If an activity is perceived as identity congruent, difficulties are interpreted as important and meaningful challenges, leading to more engagement. Conversely, if an activity is perceived as incongruent with one’s identity, the task loses meaning and may be construed as “not for people like me.” By reflecting on the personal importance of independence, FG students may counteract the effects of cultural mismatch by activating an identity that is more congruent with the university culture, resulting in increased motivation, stronger perceptions of academic fit, and ultimately, better academic performance.
Overview of Studies
In their original study, Harackiewicz, Cannning, et al. (2014) found that, relative to FG students in the control condition, FG students who were asked to affirm their most important values obtained better grades in their introductory biology course, as well as in other courses taken that semester, and were significantly less concerned about their academic background at the end of the semester. The goals of the present research were (a) to examine whether these performance effects observed in an introductory biology course persisted over subsequent semesters (Study 1A), (b) to examine whether either independence or interdependence, measured with multiple methods, accounted for the effect of the intervention on performance and on students’ concern about their background (Study 1B and 1C), and (c) to test our hypotheses experimentally by manipulating the extent to which students wrote about independence and interdependence in VA essays and testing the effects on performance on a math test (Study 2). We hypothesized that the VA effects first observed by Harackiewicz, Cannning, et al. (2014) would be long-lasting, and that these effects would be mediated by writing about independence. Furthermore, we hypothesized that writing about independence would have a causal effect on performance in the laboratory study.
Study 1A: Testing Long-Term Effects of the Values-Affirmation Intervention
Researchers posit that VA interventions have long-lasting effects on academic performance because VA has the potential to trigger positive and reciprocally reinforcing outcomes (Cohen & Sherman, 2014). That is, VA can lead to better performance, and this improved performance may further affirm the self, leading to still better performance, thus building on itself and creating a recursive process. Previous research has shown that underrepresented minority middle school students who received a VA intervention continued to earn higher grades than their peers in the control group two or three years after the initial intervention (Cohen, Garcia, Purdie-Vaughns, Apfel, & Brzustoski, 2009; Sherman et al., 2013) but such long-term VA effects have never been demonstrated in college. We predicted that the VA effects first observed by Harackiewicz, Cannning, et al. (2014) would persist such that FG students in the VA condition would obtain higher subsequent grades than FG students in the control condition. We examined students’ academic performance after the semester in which VA was implemented in two separate analyses. We first conducted a sequential time-course analysis examining grades in each of the three semesters following the intervention (a time period in which the majority of the original Harackiewicz, Cannning, et al. (2014) sample was still enrolled in college).
We then conducted longer-term follow-up analyses, three years after the intervention, and examined students’ final or current post-intervention GPA. For most students, this was their post-intervention GPA at graduation, for others, it was their final post-intervention GPA prior to dropping out or transferring from the university, and for the remaining students, it was their current post-intervention GPA, because they were still enrolled in college, three years after the intervention. These follow-up analyses allowed us to examine academic performance for students in the three semesters immediately after the intervention and over a three-year period.
Method
In the original Harackiewicz, Cannning, et al. (2014) study, 798 students were blocked on generational status (644 CG students, 154 FG students), underrepresented minority status (737 Majority students, 61 underrepresented minority students), gender (478 women, 320 men), and lecture section and then randomly assigned to complete either a VA writing exercise or a control writing exercise as part of an undergraduate biology course. The writing assignments were integrated into the laboratory curriculum as practice writing exercises and administered twice, during the 2nd and 8th weeks of the 15-week semester. Students in the VA condition were instructed to circle their two or three most important values and then write about why those values were important to them. Students in the control condition were instructed to circle the two or three values that were least important to them and to write about why other people might hold those values. The values were: athletic ability; being good at art; belonging to a social group (such as your community, racial group, or school club); career; creativity; government or politics; independence; learning and gaining knowledge; music; relationships with family and friends; sense of humor; and spiritual or religious values. These materials were based on those used in previous studies (e.g., Miyake et al., 2010); full methodological details are reported in Harackiewicz, Cannning, et al. (2014). Harackiewicz, Cannning, et al. (2014) found that FG students in the VA condition obtained higher grades in the biology course, as well as higher grades in their other courses that semester.
In the present study, we collected follow-up data to examine whether the intervention implemented in fall 2011, had long-term effects. In fall 2014, we conducted follow-up analyses in which we examined both semester grades over time and overall post-intervention GPAs. In the time-course analysis of non-cumulative semester grades, we computed students’ average grades for each of the 3 semesters following the one in which the VA intervention was implemented: spring 2012 (s1), fall 2012 (s2), and spring 2013 (s3). These were the semesters in which the vast majority of our original sample was still enrolled in classes. The number of students still enrolled in college diminished each semester after the intervention semester, due to graduation and dropping out. Thus it was not possible to extend time-course analyses further without compromising the sample. By focusing our time-course analyses on the three semesters following the one in which VA was administered, we were able to retain 94% of our original sample.1
For our analyses of post-intervention GPA, however, we were able to retain 99% of the original sample. Only ten students from the original sample did not take any more courses at this university after the intervention, meaning that we could compute a post-intervention GPA for 788 out of the 798 students in the original Harackiewicz, Cannning, et al. (2014) sample. For students who had graduated (64%), we used their post-intervention GPA at graduation; for students currently enrolled (30%), we used their current post-intervention GPA, and for students who had dropped or transferred (6%), we used their most recent post-intervention GPA. Thus we examined the most current or final post-intervention GPA for all students over a three-year time period.
Measures
Grades at this university are on a 4.0 scale (A = 4.0, AB = 3.5, B = 3.0, BC = 2.5, C = 2.0, D = 1.0, F = 0). GPAs were calculated by dividing the total number of grade points awarded to students by the total number of credits taken (including F credits) for all courses taken after the intervention semester. Concern about background was measured at the beginning and end of the intervention semester with a single item (“I am not sure I have the right background for this course”) on a 7-point scale ranging from not at all true to very true.
Samples
To examine semester grades in the three semesters following the one in which VA was implemented (fall 2011), we used grades for all students who took classes for at least two of those three semesters. This sample includes 749 of the 798 students in the original study. The 49 students not included in these analyses either graduated by June 2012 (4 students), took 2 or more semesters off during the 3 semesters following fall 2011 (11 students), or had dropped out or transferred by spring of 2013 (34 students).2 In addition to the 49 students not included in this analysis, there were also 17 students who enrolled in only two of the three semesters (i.e., they took a single semester off); for these students, we used multiple imputation to estimate their GPAs for the semester in which they were not enrolled.3 Thus, our sample for semester grade analyses consisted of 608 CG students (304 in the control condition and 304 in the VA condition) and 141 FG students (69 in the control condition and 72 in the VA condition).
For the three-year follow-up analyses, our sample consisted of 788 of the original 798 students in the Harackiewicz, Cannning, et al. (2014) study. Ten students did not take any courses after the fall 2011 semester and were therefore not included in any follow-up analyses. We computed a post-intervention GPA for these 788 students: a post-intervention final GPA for students who had graduated since fall 2011 (n = 505), a post-intervention final GPA for students who dropped out or transferred (n = 48) and a current post-intervention GPA for students still enrolled at the university (n = 235).4 The final sample for post-intervention GPA analyses consisted of 638 CG students (315 in the control condition and 323 in the VA condition) and 150 FG students (74 in the control condition and 76 in the VA condition).
Semester Grades Over Time
To explore how VA effects unfolded over time, we conducted a time course analysis examining students’ semester grades in the three semesters immediately following the intervention. We conducted a 2 (condition: control vs. VA) × 2 (generational status: CG students vs. FG students) × 3 (Time: spring 2012 semester [s1] vs. fall 2012 semester [s2] vs. spring 2013 semester [s3]) mixed-model ANCOVA with repeated measures on Time and gender as a covariate. Because students took the introductory biology course at different points in their academic careers, and had different numbers of academic credits left to take before graduation, we also controlled for their year in school when the course was taken (e.g., freshman, sophomore, junior or senior) in any analyses with post-intervention academic performance as a dependent variable.5 All possible interactions between condition, generational status, and gender were also tested; none of the gender interaction terms were significant and therefore all were subsequently trimmed from the model.
The analysis yielded a significant main effect of generational status, F(1, 745) = 4.29, p = .04, which indicated that, on average, FG students’ semester grades (M = 3.11, SD = 0.60) were lower than the GPAs of CG students (M = 3.29, SD = 0.57). A main effect of gender indicated that females had higher semester grades (M = 3.30, SD = 0.51) than males (M = 3.17, SD = 0.68), F(1, 745) = 11.97, p < .001, and a main effect of time indicated that, on average, students’ grades improved over the course of the three semesters (Ms = 3.20S1, 3.24S2, 3.32S3; SDs = 0.54S1, 0.63S2, 0.58S3). A two-way interaction between time and condition emerged, F(1, 745) = 4.82, p = .03; however this interaction was qualified by a significant three-way interaction (Time × Condition × Generational Status), F(1, 745) = 4.75, p = .03. In order to decompose the three-way interaction, simple effects tests compared the effect of condition over time first for FG students and then for CG students. These analyses revealed that over time, FG students’ semester grades were significantly higher in the VA condition (Ms = 3.11S1, 3.15S2, 3.27S3; SDs = 0.50S1, 0.60S2, 0.53S3) than in the control condition (Ms = 3.07S1, 2.99S2, 3.04S3; SDs = 0.52S1, 0.74S2, 0.70S3), F(1, 745) = 5.89, p = .02. Conversely, CG students’ grades did not vary over time as a function of VA condition, F(1, 745) = 0.00, p = .99 (within VA: Ms = 3.21S1, 3.27S2, 3.35S3; SDs = 0.58S1, 0.64S2, 0.56S3; within control: Ms = 3.24S1, 3.28S2, 3.37S3; SDs = 0.50S1, 0.59S2, 0.56S3). Figure 1 shows the effects of VA on semester grades over time reported here, in the context of the original VA effect. Specifically, it shows that students’ academic performance did not differ by condition on pre-intervention GPA, but did differ in the semester in which VA was implemented (fall 2011) and in the three subsequent semesters as documented above; FG students in the VA condition earned higher grades than FG students in the control condition.
Figure 1.
GPAs with +/−1 standard error for performance among continuing-generation (CG) and first-generation (FG) students as a function of values affirmation (VA) condition; N = 749. Fall 2011 is the semester in which VA was implemented. Pre-intervention and post-intervention grade point averages (GPAs) are cumulative. Fall 2011, Spring 2012, Fall 2012, and Spring 2013 are semester GPAs.
Post-Intervention GPA
Our basic regression model tested the effects of condition (control = −1, VA intervention = 1), generational status (CG students = −1, FG students = 1), the condition by generational status interaction, gender (females = −1, males = 1), and year in school when the biology course was taken. All possible interactions between condition, generational status, and gender were tested; none of the gender interaction terms were significant and therefore all were subsequently trimmed from the model.
Regression analyses indicated that there was a main effect of generational status such that CG students obtained higher cumulative post-intervention GPAs (M = 3.27, SD = 0.54) than FG students (M = 3.07, SD = 0.65); t(782) = 4.31, p < .001, β = −0.15, revealing a social class achievement gap in post-intervention GPA. However, this main effect was qualified by the predicted interaction with VA condition, t(782) = 2.22, p = .03, β = 0.10, indicating that FG students in the VA condition (M = 3.16, SD = 0.58) obtained higher post-intervention GPAs than FG students in the control condition (M = 2.98 SD = 0.70). CG students performed similarly in the VA condition (M = 3.25 SD = 0.57) and the control condition (M = 3.28 SD = 0.49). Whereas the achievement gap was moderate in the control condition (0.31 GPA points), Cohen’s d = .33, t(782) = 4.59, p < .001, it was substantially smaller (and non-significant) in the VA condition (.10 GPA points), Cohen’s d = .11, t(782) = 1.50, p = .14, reflecting a treatment effect of .18 grade points, resulting in a 59% reduction of the social class achievement gap in post-intervention cumulative GPA. Finally, a main effect of gender indicated that females had significantly higher post-intervention cumulative GPAs (M = 3.30, SD = 0.49) than males (M = 3.13, SD = 0.65), t(782) = 4.58, p < .001, β = −0.16. Figure 2 shows the original VA effect on course grade as well as the effect on post-intervention cumulative GPA documented here; Table 1 shows the full regression model for this analysis.
Figure 2.
Mean biology course grade (Panel A) and post-intervention GPA (Panel B) with +/−1 standard error for performance among continuing-generation and first-generation students as a function of values affirmation condition; N = 798 in Panel A; N = 788 in Panel B.
Table 1.
Study 1A: Regression Analysis of Cumulative Post-intervention GPA
| Regression | ||||||||
|---|---|---|---|---|---|---|---|---|
| Model 1 | Model 2 | |||||||
| B | β | t (df) | p | B | β | t (df) | p | |
| Post-Intervention GPA | ||||||||
| Condition | .04 | 0.07 | 1.57 (1, 782) | .118 | .04 | 0.08 | 2.12 (1, 781) | .034 |
| Generational Status | .11 | −0.15 | 4.31 (1, 782) | .000 | .07 | −0.09 | 3.10 (1, 781) | .002 |
| Condition × Generational Status | .06 | 0.10 | 2.22 (1, 782) | .027 | .05 | 0.09 | 0.23 (1, 781) | .013 |
| Gender | .09 | −0.16 | 4.58 (1, 782) | .000 | .07 | −0.19 | 4.42 (1, 781) | .000 |
| Year In School | .01 | −0.13 | 0.38 (1, 782) | .704 | .01 | 0.01 | 0.25 (1, 781) | .800 |
| Performance Covariate | -- | -- | -- | -- | .30 | 0.55 | 18.65 (1, 781) | .000 |
Note. Two sets of regressions examining effects on post-intervention GPA are presented here. Model 1 displays the in-text reported effects on post-intervention GPA. Model 2 shows these same effects but also includes a baseline covariate (a standardized composite measure of students’ prior semester GPA, ACT, and SAT) that was used in the Harackiewicz, Canning et al. (2014) original study.
Discussion
The results of Study 1 indicate that the VA intervention had long-lasting effects on academic performance for FG students. This finding is consistent with long-term effects documented by Cohen et al. (2009) and Sherman et al. (2013) in middle-school populations, but it is the first demonstration of a long-term VA effect with FG college students. Indeed, the brief VA intervention, implemented twice in an early foundational course, had far-reaching effects that may have changed students’ academic trajectories. Three years after the intervention, FG students in the VA condition, on average, had maintained a GPA .18 grade points higher than FG students in the control condition. This moderate boost in GPA might increase the options for graduate education or career training, especially considering that average performance for FG students rose from the B range in the control condition (M = 2.98) to the B+ range in the VA condition (M = 3.16). FG students who may otherwise have fallen just below an important performance cutoff (i.e., 3.0 GPA) may have been able to cross that academic barrier and fulfill their potential because of the VA intervention.
The analyses on semester grades provide insight regarding how the social-class achievement gap may unfold over time. For most students in the current sample (i.e., CG students and FG students in the VA condition) and consistent with patterns of college performance rates (Betts & Morell, 1999; Geiser & Santelices, 2007; Grove & Wasserman, 2004; UW-Registrar, 2015), semester grades improved over time. As students become acclimated to college they may gain a better understanding of the skills and study habits necessary for successful degree completion. However, this pattern of improving performance is not seen in the GPA trajectories of FG students in the control condition. These FG students did not improve at the same rate as CG students, and in some cases, their semester grades even worsened relative to previous semesters. It may be that values affirmation helps students adjust to the ongoing challenges of college coursework, and our results suggests that intervention early in the course of students’ academic careers may yield long-term benefits. Given the power of the VA intervention to influence performance in the short and long term, it is important to examine the mechanisms through which it works.
Study 1B: Exploring Mediation Through Content Analyses
Study 1B involved new analyses and coding of the essays written by students in the Harackiewicz, Cannning, et al. (2014) study. Harackiewicz, Cannning, et al. (2014) noted that FG and CG students did not select different values to write about in the VA condition, indicating that the positive VA effects were not driven by differential selection of values. Thus, in the present study, we examined the content of students’ essays in order to identify if VA effects were driven by differences in how students wrote about their selected values. Specifically, VA essays were coded for themes of independence and interdependence so that these two constructs could be tested as mediators of the positive effects of VA for FG students. We also examined the interrelationship of independent and interdependent themes in VA essays to examine the extent to which students wrote about independence, interdependence, or a combination of both in their essays. All essays (both control and VA) were first coded by the research team (Study 1B). We then conducted a linguistic text analysis of the VA essays (Study 1C) to further explore causal mechanisms.
Holistic Coding of Interdependent and Independent Writing
All essays were coded using a binary coding scheme for the presence or absence of interdependent and independent themes. The criteria for coding interdependent themes were identical to those employed by Shnabel et al. (2013) for social belonging,6 defined as: an explicit mention of (1) valuing an activity because it is done with others, (2) feeling part of a group of people because of a certain value or while engaging in a certain activity, or (3) any related thoughts on the subject of one’s interdependence, such as being affiliated with or liked by others.
We developed a new coding system for independent themes using similar guidelines. Specifically, independent themes were defined as (1) valuing an activity because it is done alone, (2) explicitly expressing the value of independence for the self, or (3) any related thoughts showing that the participant values his or her own autonomy (i.e., the ability to make her or his own decisions and have his or her own ideas and opinions).
Each essay was coded for independent and interdependent themes by at least two trained coders, and we maintained high inter-rater reliability for both independence (Cohen’s kappa = .96) and interdependence (Cohen’s kappa = .89) coding (Landis & Koch, 1977). Initial agreement among coders was over 90%, and for the few instances in which coders did not agree, a third coder was consulted to resolve any ambiguity. Because participants wrote two essays over the course of the semester, they received two scores for independence and two scores for interdependence. Each pair of scores was summed to create a single score that reflected the extent to which students wrote about independent and interdependent themes across the two essays (i.e., scores could be 0, 1, or 2 for each measure).
Although students were not asked to reflect on personal values in the control condition (rather, they were prompted to write about why their least important values might be valued by others), there were instances (10% of control essays) in which students wrote about their own independence, interdependence, or both in the control condition. This usually occurred when participants commented on their least important values by contrasting them with what they actually valued. For example, one control participant discussed their own independence by stating “Belonging to a social group may be important to someone else. I personally like to be more independent.” Similarly, several control essays included themes of interdependence, for example “There are people who live for art and their ability to be creative, I am just not one of them. Being with friends and family and religion are my top priorities.” Given the presence of self-relevant independent and interdependent themes in the control condition, we coded for both themes in both conditions.
Results
Table 2 provides sample quotes from essays that were coded as independent, interdependent, or both interdependent and independent. Across conditions, 10% of participants wrote about independence once (19% in VA conditions), 8% wrote about independence twice (17% in VA), and 82% of participants never wrote about independence (64% in VA). For interdependence, 11% wrote about interdependence once (14% in VA), 40% wrote about interdependence twice (78% in VA), and 49% never wrote about interdependence (8% in VA).
Table 2.
Study 1B: Sample Quotes from Values Affirmation Essays
| Independence | Interdependence | Both |
|---|---|---|
| Independence and learning and gaining knowledge are very important values to me. These are a few of the many reasons why I chose to go to college and further my education. The first full day at college after moving in was a dream come true. For once I finally felt independent and I had full control over my life and my future. Gaining knowledge and learning new things are very important. It means that I will be able to support myself in the future by having a successful career and being able to do the thing I want to in life. It makes me feel more important and that I am accomplishing something that means a lot. After graduating college, it will feel great to be on my own truly for once. By having a good job that pays well is important to me. It will allow me to pursue my hobbies and interests. Independence is a big one, especially in college, because it can be the difference between sinking and swimming. Being independent means being able to work or study without instruction, and being able to meet goals with very little external motivation. Without independence, college is basically meaningless. Being independent is an aspect of my life is of upmost importance to me. Choosing to attend college away from home was a large step in autonomy; living on my own, and making my own choices were large but necessary changes in my life in order for me to become independent and grow up into an adult. Also, the ability to study whatever I choose gave me great autonomy to explore my interests and learn about a wide variety of studies. Thus I have acquired more independence and knowledge with my transition into college. Acquiring these skills has made me feel successful as a young adult and student, I believe I gained a lot from this large transition from high school to college. Belonging to a social group may be important to someone else. I personally like to be more independent but there is nothing wrong with belonging to a social group and I’m sure it is a lot of fun for others. (control condition) |
Friends and family are extremely important to me. Having good relationships with them is essential to living a happy life. They’re the people who can provide me with help, empathy and condolence in times of sadness and need. They also are the people who make life fun because they’re the people I love. Belonging to a social group is important because you’re surrounding yourself by people who you have something in common with. I think it’s always beneficial to have people like this when friends and family can’t necessarily help you out. Talking or going to these people is a nice alternative support/ help group to have. My relationships with friends and family is definitely very important to me. These are the people who I go to for support and guidance. Knowing that I can count on them and they can count on me makes my life happier and more relaxed. Also, along with this having a social group to belong to is important. Being part of a team or club makes me feel like I am a part of something and that people count on me to do my best. These teams and clubs help me become a better person and allow me to be with people who are passionate about the same things as I am. I believe in being with my family and friends. I am happiest when I’m with them so it is very important to me. I love my friends and family and I don’t want to be in a world without them. I love hanging out with my friends and my family because it creates togetherness and stronger relationships. There are people who live for art and their ability to be creative, I am just not one of them. Being with family and friends and religion are my top priorities.” (control condition) |
I chose independence because I have learned in my previous year here at UW that independence is very important. I lived in Witte Hall last year so I was in the center of the social community of college students. There were always groups of people at the dorms who would get together for playing games, hanging out, or studying. I enjoyed participating in all of these activities but I realized that I need to dedicate more time to studying by myself and getting some alone time. After dedicating more of my schedule to independent time, I began to understand material better and got to know myself and my goals better. I have been working in a neuroscience research lab for the last two years. At first, I felt pretty overwhelmed by all of the knowledge and information I’d need to learn about the lab, but as time went on my curiosity drove me to learn and gain this knowledge. I was no longer dependent on my lab manager for every little piece of information. This independence in lab has really made me feel successful and self-sufficient in lab. I now run my own experiments and train new students who were in the exact same shoes as I was two years ago. I also have a friend who really cares about belonging to a social clique in her school. Honestly, I don’t care much for it because I have taught myself to be more independent and not rely on others to help me go far in life. I do value friendships, however, I just don’t stress about belonging to a specific group. (control condition) |
Notably, participants in the VA condition often wrote about both their independence and interdependence in their essays (see Table 3 for percentages of students who wrote about independence and interdependence in VA conditions). Furthermore, very few students wrote exclusively about independence; rather, the majority of students wrote about interdependence, and when students wrote about independence, it was generally in addition to writing about interdependence. In fact, of the participants who wrote about independence, 84% also wrote about interdependence, suggesting that writing about independence does not necessarily come at the expense of writing about interdependence. Importantly, this pattern of results did not differ as a function of generational status, p = .16.
Table 3.
Study 1B: Percentage of Students in the Values-Affirmation Condition Who Wrote About Independence and Interdependence
| Generational Status |
No Independence or Interdependence |
Only Independence |
Only Interdependence |
Both Interdependence and Independence |
|---|---|---|---|---|
| CG Students | 4% | 6% | 62% | 28% |
| FG Students | 0% | 2% | 56% | 42% |
| All Students | 3% | 5% | 61% | 31% |
Note. CG = continuing-generation; FG = first-generation.
Regression Model
For models examining dependent variables that were measured in the semester in which VA was implemented (i.e., independent and interdependent themes, course grade, concern about background) we tested the basic model described earlier but included the same factors and interaction terms that were tested by Harackiewicz, Cannning, et al. (2014). Thus, unless otherwise specified, any analysis of dependent variables measured within the semester in which VA was implemented tested for the effects of condition (control = −1, VA = 1), generational status (CG students = −1, FG students = 1), the condition by generational status interaction, and gender (females = −1, males = 1), and also controlled for lecture section (two orthogonal codes to control for differences between the three sections of the class) and two interactions between generational status and lecture section.7 Gender interactions were initially tested in each analysis but then trimmed from all models as none were significant, p > .18.
Effect of VA Condition on Interdependent and Independent Themes
In order to test interdependent and independent writing as potential mediators of the positive VA effect for FG students, we first tested whether students in the VA condition wrote more about independence and/or interdependence in their essays than students in the control condition. We found a significant main effect for treatment condition on both independent and interdependent themes, t(789) = 11.08, p < .001; β = 0.45, and t(789) = 37.78, p < .001, β = 0.81, respectively, indicating that students wrote about both independence and interdependence significantly more often in the VA condition compared to the control condition, as expected. We also found a main effect of generational status indicating that FG students (M = 0.98, SD = 0.93) wrote more often about interdependence than CG students (M = 0.88, SD = 0.94), t(789) = 2.01, p = .04, β = 0.04. The frequency of writing about independent themes did not vary by generational status (p = .28) and there were no significant interactions between treatment condition and generational status on either independence (p = .23) or interdependence (p = .32). A significant main effect of gender also emerged indicating that females (M = 0.98, SD = 0.95) wrote more often about interdependence that males (M = 0.77, SD = 0.91), t(789) = 5.19, p < .001, β = −0.09.
Moderated Mediation Model
Figure 3 illustrates the mediation model tested. We used Hayes’ (2013) PROCESS software, which allowed us to test the indirect effects of VA on performance and concern about background through independent and interdependent themes as a function of generational status (i.e., a × b; and d × e; see Figure 3). In other words, these analyses allow us to examine if writing about independence or interdependence mediated VA effects on outcome variables (course grade, post-intervention GPA, and concern about background) and test if mediation effects were moderated by generational status. We predicted that any mediation effects would be moderated by generational status such that the mediator (writing about independence or writing about interdependence) would be a particularly strong predictor of intervention effects for FG students.
Figure 3.
Moderated Mediation Model.
Course grades
Table 4 summarizes the effects of VA condition on independent and interdependent themes, as well as the effects of VA, generational status, and the hypothesized mediators on final course grade. The “Conditional Indirect Effects” section of Table 4 shows the significance tests for the indirect effects of independent themes and interdependent themes as a function of generational status on course grade. The “Index of Moderated Mediation” tests if the indirect effect through each of the proposed mediators varied as a function of generational status. Given that the confidence interval for the index of moderated mediation for independent themes did not include zero (95% CI: = [0.028, 0.168]), we can conclude that the indirect effect of independence on course grade varied significantly as a function of generational status. Specifically, independent writing was a significant mediator for FG students (95% CI = [0.035, 0.167]) but was not a significant mediator for CG students (95% CI = [−0.031, 0.033]). Thus, students assigned to the VA condition wrote more often about independence than students in the control condition, and for FG students, writing about independence was associated with higher grades in the class.8 There were no significant indirect effects for interdependent themes, indicating that it was not a mediator for either CG or FG students.
Table 4.
Study 1B: Moderated Mediation of Effects of the Intervention on Course Grade
| Predictor | B | SE | t | p | |
|---|---|---|---|---|---|
| DV: Biology 151 Course Grade | |||||
| VA Intervention | −.001 | .074 | −.001 | .994 | |
| Generational status | −.260 | .045 | −5.719 | .000 | |
| Independent Themes | .117 | .049 | 2.402 | .017 | |
| Interdependent Themes | .002 | .078 | .026 | .979 | |
| Independence × Generational status | .114 | .047 | 2.430 | .015 | |
| Interdep × Generational status | .025 | .048 | .519 | .604 | |
| Index of Moderated Mediation | |||||
| Mediator | Index | Boot SE | Boot LLCI | Boot ULCI | |
| Independent Themes | .095 | .037 | .028 | .168 | |
| Interdependent Themes | .042 | .094 | −.142 | .227 | |
| Conditional Indirect Effects | |||||
| Mediator | Generational status |
Boot indirect effect |
Boot SE | Boot LLCI | Boot ULCI |
| Independence | CG Students | .002 | .016 | −.031 | .033 |
| Independence | FG Students | .097 | .033 | .035 | .167 |
| Interdependence | CG Students | −.020 | .061 | −.141 | .099 |
| Interdependence | FG Students | .023 | .103 | −.182 | .223 |
Note. N = 798 (644 CG students, 154 FG students). Confidence intervals are reported with a bootstrap sample size = 5000. LLCI = lower level of the 95% bootstrap percentile confidence interval; ULCI = upper level of the 95% bootstrap percentile confidence interval. Condition and generational status were coded such that Control = −1 and VA = 1; similarly, CG students = −1 and FG students = 1. Testing each mediator separately revealed conceptually analogous results.
Post-intervention GPA
In the model testing for moderated mediation of post-intervention GPA, the confidence interval for the index of moderated mediation for independent themes did not include zero (95% CI = [0.041, 0.190]), indicating that the indirect effect of independent writing on post-intervention GPA varied as a function of generational status. Specifically, independent writing was a significant mediator for FG students (95% CI = [0.024, 0.155]) but was not a significant mediator for CG students (95% CI = [−0.069, 0.011]); see on-line supplemental materials for full moderated-mediation table. Thus, students assigned to the VA condition wrote more often about independence than students in the control condition, and for FG students, writing about independence was associated with better academic performance (i.e., higher GPAs) three years later. There were no significant indirect effects for interdependent themes, indicating that it was not a mediator for either CG or FG students.
Concern about background
Table 5 shows the effects of VA condition on independent and interdependent themes, as well as the effects of VA, generational status, and the hypothesized mediators on students’ concern about their background at the end of the semester. Because the effect on student’s concerns emerged over time in the Harackiewicz, Cannning, et al. (2014) study, we controlled for baseline levels of concern in these models, for consistency with the original analyses. Given that the confidence interval for the index of moderated mediation for independent themes did not include zero (95% CI = [−0.149, −0.013]), we can conclude that the indirect effect of independent writing on students’ concern varied as a function of generational status. Specifically, independent writing was a significant mediator for FG students (95% CI = [−0.132, −0.009]), but was not a significant mediator for CG students (95% CI = [−0.026, 0.046]). Thus, FG students assigned to the VA condition wrote more often about independence than FG students in the control condition, and writing about independence was associated with less concern about their academic background over time (indirect effect b = −.07 for FG students, compared to b = .01 for CG students). There were no significant indirect effects for interdependent themes, indicating that it was not a mediator for either CG or FG students.
Table 5.
Study 1B: Moderated Mediation of the Effects of the Intervention on Concern About Background
| Predictor | B | SE | t | p | |
|---|---|---|---|---|---|
| DV: Concern about Background | |||||
| VA Intervention | −.053 | .067 | −.785 | .433 | |
| Generational status | .085 | .041 | 2.042 | .042 | |
| Independent Themes | −.070 | .045 | −1.558 | .120 | |
| Interdependent Themes | .055 | .071 | .771 | .441 | |
| Independence × Generational status | −.094 | .043 | −2.198 | .028 | |
| Interdepend. × Generational status | −.046 | .043 | −1.052 | .293 | |
| Baseline Concern about Background | .430 | .032 | 13.369 | .000 | |
| Index of Moderated Mediation | |||||
| Mediator | Index | Boot SE | Boot LLCI | Boot ULCI | |
| Independent Themes | −.078 | .035 | −.149 | −.013 | |
| Interdependent Themes | −.079 | .085 | −.244 | .089 | |
| Conditional Indirect Effects | |||||
| Mediator | Generational status |
Boot indirect effect |
Boot SE | Boot LLCI | Boot ULCI |
| Independence | CG Students | .010 | .018 | −.026 | .046 |
| Independence | FG Students | −.068 | .031 | −.132 | −.009 |
| Interdependence | CG Students | .086 | .054 | −.019 | .195 |
| Interdependence | FG Students | .007 | .093 | −.173 | .191 |
Note. N = 796 (642 CG students, 154 FG students). Confidence intervals are reported with a bootstrap sample size = 5000. LLCI = lower level of the 95% bootstrap percentile confidence interval; ULCI = upper level of the 95% bootstrap percentile confidence interval. Condition and generational status were coded such that Control = −1 and VA = 1; similarly, CG students = −1 and FG students = 1. Testing each mediator separately revealed conceptually analogous results.
Decomposition of moderated mediation effects
Our models tested mediating variables measured on a continuous scale (independent and interdependent writing were each coded on a 0–2 scale), as recommended by Hayes (2013). To depict the effect graphically, however, we recoded the three-level independent themes measure as a dichotomous measure in the VA condition and categorized students as never having written about independence versus those who did write about independence, either once or twice. Figure 4 shows this decomposition of the treatment effects on course grade, post-intervention GPA, and concern about background. The first panel shows results in the control group. The second and third panels in each graph represent two groups of students in the VA condition, those who never wrote about independence and those who did. We tested regression models with dummy codes using the control group for comparison, to examine performance and concern for students who did and did not write about independence in their VA essays. A significant interaction between generational status and the dummy code comparing students who affirmed their independence in VA to the control condition indicated that FG students performed significantly better in the class, t(787) = 2.78, p <. 01, β = 0.13, attained higher post-intervention GPAs, t(780) = 3.11, p < .01, β = 0.15, and were less concerned about their background t(784) = 2.49, p = .01, β = −0.11, when they affirmed their independence in the VA condition. Conversely, there were no significant differences between FG students who did not affirm their independence in VA and FG students in the control condition, p > .35.
Figure 4.
Study 1B: Decomposition of moderated mediation effects comparing students in the values affirmation (VA) condition at different levels of the mediator - those who wrote about independence (VA, Independence) and those who did not write about independence (VA, No Independence) - to students in the control condition on mean biology course grade (Panel A), post-intervention GPA (Panel B), and concern about background (Panel C). Error bars represent +/−1 standard error.
These results are consistent with the moderated mediation analyses reported earlier. We found consistent results indicating that the positive effect of VA for FG students was mediated through writing about independence and moderated by generational status: FG students (but not CG students) benefited from writing about their personal independence across three measures (course grade, post-intervention GPA, and concern about background). In other words, the intervention promoted writing about independence for all students (i.e., all students wrote more about independence in the VA condition), on average, but this writing was particularly powerful for FG students.
Although these results are consistent with our hypotheses, for a more complete understanding of this moderated mediation, it is important to consider whether other factors were associated with writing about independence for FG students. For example, a previously unexplored third variable might predict both writing about independence and academic performance for FG students (Imai & Yamamoto, 2013)9. It is therefore important to examine whether there were differences on baseline variables for FG students who chose to write about independence compared to those who did not, and then test whether any such differences might account for the moderated mediation effects documented here.
Exploring possible confounding variables
We examined several baseline measures for differences between FG students who wrote about independence (and benefited most from the intervention) and those who did not write about independence. An effect of independent writing on a baseline variable could indicate that it was a pre-existing individual difference and not the effects of the intervention that caused some FG students to write more about independence, perform better, and feel less concern about background.
We tested a regression model that included the main effects of condition (Control vs. VA), generational status (FG students vs. CG students), and independence code (wrote about independence vs. never wrote about independence) as well as all two-way interactions on the following baseline measures: percentage of students who receive free or reduced lunch at each student’s high school (a proxy for poverty at both the school and neighborhood level), a measure of pre-intervention performance (prior GPA), a standardized test score (ACT), number of total credits taken in the previous semester, confidence about performance, baseline concern about background, and age. Missing data was addressed using multiple imputation. Due to the low level of independence affirmed in the control condition, the 3-way interaction between condition, generational status, and independence was not included, given collinearity with the generational status × independence interaction term. For these analyses, it is critical to identify any effects that include independence as a significant predictor as this could indicate baseline differences between students who did and did not write about independence and introduce a potential confound for the moderated mediation findings.
The results of the baseline analyses are displayed in Table 6. The results show that there were no failures of randomization on baseline measures (i.e. no condition main effects or interactions with condition on any baseline measures). They also indicate that FG students differed from CG students on a number of variables at baseline. Specifically, FG students attended high schools with a higher percentage of students receiving financial assistance for school meals (FG students: M = 31%, SD = 16%; CG students: M = 22% SD = 15%), obtained, on average, lower ACT scores (FG students: M = 27.58, SD = 3.31; CG students: M = 28.60, SD = 2.61), had lower prior GPAs (FG students: M = 3.10, SD = 0.53; CG students: M = 3.23, SD = 0.50), and were older (FG students: M = 19.70 years old, SD = 2.08; CG students: M = 19.16 years old, SD = 0.82) than their CG peers.
Table 6.
Study 1B: Analysis of Baseline Variables
| Regression | ||||||
|---|---|---|---|---|---|---|
| Step 1 | Step 2 | |||||
| β | t (df) | p | β | t (df) | p | |
| % Free/Reduced Lunch | ||||||
| Generational Status | 0.22 | 6.41 (794) | .000 | 0.20 | 4.39 (791) | .000 |
| Condition | 0.01 | 0.24 (794) | .811 | 0.12 | 0.13 (791) | .898 |
| Gen Status × Condition | −0.03 | 0.66 (794) | .509 | −0.03 | 0.62 (791) | .951 |
| Independence | −0.40 | 0.42 (791) | .676 | |||
| Gen Status × Independence | −0.05 | 0.89 (791) | .376 | |||
| Condition × Independence | −0.28 | −0.26 (791) | .793 | |||
| ACT | ||||||
| Generational Status | −0.15 | 4.12 (794) | .000 | −0.05 | 1.09 (791) | .275 |
| Condition | 0.00 | 0.03 (794) | .976 | −0.13 | 1.05 (791) | .295 |
| Gen Status × Condition | 0.00 | 0.08 (794) | .939 | 0.10 | 1.87 (791) | .062 |
| Independence | 0.22 | 2.23 (791) | .026 | |||
| Gen Status × Independence | 0.20 | 3.47 (791) | .001 | |||
| Condition × Independence | −0.01 | −0.10 (791) | .918 | |||
| Prior GPA | ||||||
| Generational Status | −0.11 | 3.01 (794) | .003 | −0.06 | 1.35 (791) | .178 |
| Condition | −0.01 | 0.18 (794) | .855 | −0.09 | 0.72 (791) | .472 |
| Gen Status × Condition | 0.01 | 0.30 (794) | .767 | −0.03 | 0.57 (791) | .568 |
| Independence | 0.10 | 1.02 (791) | .307 | |||
| Gen Status × Independence | 0.09 | 1.52 (791) | .129 | |||
| Condition × Independence | −0.04 | 0.36 (791) | .716 | |||
| Prior Credits Taken | ||||||
| Generational Status | −0.04 | 1.09 (794) | .275 | −0.06 | 1.35 (791) | .176 |
| Condition | 0.02 | 0.39 (794) | .700 | 0.12 | 0.97 (791) | .333 |
| Gen Status × Condition | 0.01 | 0.21 (794) | .834 | 0.03 | 0.52 (791) | .600 |
| Independence | −0.07 | 0.72 (791) | .472 | |||
| Gen Status × Independence | −0.04 | 0.71 (791) | .477 | |||
| Condition × Independence | 0.10 | 0.93 (791) | .353 | |||
| Confidence about Performance | ||||||
| Generational Status | −0.01 | 0.21 (794) | .833 | 0.02 | 0.36 (791) | .720 |
| Condition | −0.02 | 0.50 (794) | .619 | 0.06 | 0.45 (791) | .650 |
| Gen Status × Condition | 0.02 | 0.50 (794) | .619 | 0.00 | 0.01 (791) | .993 |
| Independence | −0.43 | 0.43 (791) | .665 | |||
| Gen Status × Independence | 0.05 | 0.88 (791) | .381 | |||
| Condition × Independence | 0.09 | 0.89 (791) | .386 | |||
| Concern about Background | ||||||
| Generational Status | −0.01 | 0.37 (794) | .384 | −0.04 | 0.97 (791) | .333 |
| Condition | 0.03 | 0.72 (794) | .325 | −0.01 | 0.05 (791) | .959 |
| Gen Status × Condition | 0.00 | 0.03 (794) | .315 | 0.03 | 0.67 (791) | .500 |
| Independence | −0.02 | 0.17 (791) | .866 | |||
| Gen Status × Independence | −0.07 | 1.18 (791) | .237 | |||
| Condition × Independence | −0.08 | 0.73 (791) | .466 | |||
| Age | ||||||
| Generational Status | 0.18 | 5.16 (794) | .000 | 0.19 | 4.17 (791) | .000 |
| Condition | 0.00 | 0.06 (794) | .952 | 0.06 | 0.51 (791) | .611 |
| Gen Status × Condition | 0.00 | 0.06 (794) | .952 | −0.01 | 0.17 (791) | .865 |
| Independence | −0.06 | 0.58 (791) | .560 | |||
| Gen Status × Independence | 0.02 | 0.34 (791) | .738 | |||
| Condition × Independence | 0.05 | 0.48 (791) | .633 | |||
Note. Step 1 tests the effects of generational status, condition, and the Generational Status × Condition effect. Step 2 adds Independence, and two 2-way interactions (Generational Status × Independence and Condition × Independence).
When we tested for differences as a function of Independence code, however, we found effects only for ACT scores. Specifically, there was a significant main effect of independence code such that students who wrote about their independence had higher ACT scores (M = 28.85, SD = 2.49) than students who did not write about their independence (M = 28.30, SD = 2.85), t(791) = 2.23, p = .03, β = 0.22. Furthermore, this main effect was qualified by a generational status×independence code interaction revealing that FG students who wrote about independence (M = 29.10, SD = 2.70) had higher ACT scores than FG students who did not write about independence (M = 27.15, SD = 3.35), t(791) = 3.47, p = .001, β = 0.20, whereas the difference was much smaller for CG students: M = 28.77, SD = 2.43, for those who wrote about independence, and M = 28.57, SD = 2.65 for those who did not. These effects, observed across condition, were also significant when analyzed within the VA condition (see on-line supplemental materials for these analyses).
These effects suggest that FG students who wrote about independence in VA conditions also had stronger academic preparation in high school (as evidenced by higher ACT scores). It is possible that this stronger academic background might account both for FG students’ more independent writing and their improved performance and/or decreased concern about background. It is therefore important to test whether the effects remain significant when ACT is included in the model. We therefore tested whether differences in ACT accounted for the moderated mediation reported earlier by including ACT in our models. Specifically, we included the main effect of ACT, and three two-way interactions (ACT × Generational Status, ACT × Condition, and ACT × Independence Code). Controlling for these variables did not change the pattern of results or significance of the conditional indirect effects of independence on outcome variables for FG students (95% CI: = [0.008, 0.137] for course grade; 95% CI: = [0.003, 0.129] for post-intervention GPA; 95% CI: = [−0.200, −0.007] for concern about background); moreover, the indices of moderated mediation remained significant (95% CI: = [0.005, 0.146] for course grade; 95% CI: = [0.022, 0.171] for post-intervention GPA; 95% CI: = [−0.238, −0.013] for concern about background). In other words, even with ACT controlled, the positive effect of independent writing for FG students remained significant.10 The fact that the indicators of moderated mediation remained significant after controlling for ACT suggests that the mediating effect of writing about independence was not an artifact of higher ACT scores. In other words, although FG students with higher ACT scores were more likely to write about independence, our results indicate that this ACT difference does not account for the beneficial effect of affirming independence for FG students. These analyses bolster our conclusion that writing about independence was a critical mediator of intervention effects for FG students.
Discussion
The results of Study 1B support our hypothesis that writing about independence would mediate the positive effects of VA on course grade, post-intervention GPA, and concern about background for FG students. We found that students assigned to the VA condition wrote more about both independence and interdependence in their essays than students assigned to the control condition, with many students affirming both their independence and interdependence in the same essay (indicating that these two types of values are not mutually exclusive). However, only independent writing was associated with improved academic performance for FG students, in both the short and the long term, as well as less concern about their background.
Although we have tested independent writing as a mediator of treatment effects for FG students, it is important to consider whether independence could be a moderator rather than a mediator. For example, we conceptualized independent writing as a mediator, and demonstrated that the VA manipulation induced students to write more about their independent values, which in turn benefited FG students. Alternatively, one could argue that independent writing is a moderator such that the VA intervention induced all students to write about their values, and the FG students who chose to write about independence (perhaps based on some unobserved individual difference) benefited the most. Although this logic is plausible, we considered writing about independence to be a mediator rather than a moderator for several reasons.
Our analyses indicate that VA increased both independent and interdependent writing but that only independent writing accounted for the positive effect of VA for FG students. This way of thinking about the content of VA writing as a mechanistic outcome of VA rather than a moderator is consistent with prior research that documented how writing about social belonging mediated VA effects for African American middle-schoolers (Shnabel et al., 2013). However, we recognize that this argument is not as clear-cut as in other research where mediators are measured later in time (Smith, 2012). Ultimately, the distinction between independent writing as a mediator or moderator may rest on whether writing independently is considered to be the intervention itself or an outcome of the intervention (which instructs student to first think about important values and then write about them), and this may simply be a question of semantics. We view the content of VA essays as evidence of the psychological processes occurring as the immediate result of the intervention, and therefore consider independent writing to be an outcome and mediator of the intervention.
To strengthen this interpretation, we conducted a number of analyses to rule out potential confounds (e.g., exploration of possible confounding variables at baseline) that might have significantly predicted both the mediator (writing about independence) and outcome variables (grades, concern about background), but stronger evidence for mediation would come from a study in which the mediator was directly manipulated (Bullock, Green, & Ha, 2010; Smith, 2012). Thus, we sought to test the effects of inducing independent writing in an experimental study in Study 2. Although one can never fully rule out that an unmeasured variable accounts for the observed results (or signals that independent writing moderated our effects), we believe that our analyses support the conceptualization of independent writing as a mediator. Furthermore, we conducted linguistic analyses (Study 1C) and a controlled laboratory study (Study 2) in order to better understand the role of independent writing in VA essays.
Study 1C: Linguistic Analysis of VA Essays
A principal finding of Study 1B was that independent writing accounted for the beneficial effects of the VA intervention for FG students. However, the measures of independent and interdependent writing were three-level measures based on holistic coding. In order to examine the content of students’ writing in greater detail, as well as how they were expressing themselves when they reflected on important personal values, we conducted a text analysis exclusively within the VA condition. We used Linguistic Inquiry and Word Count (LIWC; Pennebaker, Francis, & Booth, 2001) to examine if the positive effect of writing about independence for FG students observed across conditions in Study 1B could be replicated utilizing a more detailed content analysis that examined the independent and interdependent linguistic content of VA essays.
LIWC coding
Given that previous analyses revealed that FG and CG students did not select different values in the VA condition (Harackiewicz, Cannning, et al., 2014), we used a text analysis program to focus on the content of students’ essay in greater detail and examined if FG and CG students wrote about their values differently or if certain kinds of writing were particularly beneficial for FG students. Thus, we conducted a content analysis that enabled us to examine, with greater nuance, the extent to which students wrote about independence and interdependence. We developed text analytic dictionaries starting with Madera, Hebl, & Martin’s (2009) Agentic and Communal dictionaries and then added more terms, based on existing literature on independent and interdependent individual orientations and writing styles (e.g., Markus & Kitayama, 1991; Stephens, Fryberg et al., 2012) to further guide the dictionary construction process. Words included in the independent dictionary included themes of individual interest and achievement, self-discovery, uniqueness, and leadership. Words in the interdependent dictionary reflected interpersonal themes of belonging, family, support, and empathy (see on-line supplemental materials for full independent and interdependent dictionaries).
Results
We analyzed all of the VA essays from the Harackiewicz, Cannning, et al. (2014) study with LIWC, which yielded scores for the percentage of independent and interdependent words used in each essay. Because students wrote two VA essays over the course of the semester, we averaged the independent scores to create a single independent words score and averaged the interdependent scores to create an interdependent words score. On average, students wrote 135.39 words in their essays with 4% (SD = 2%) independent words and 7% (SD = 3%) interdependent words. There was a small negative correlation between independent and interdependent words, r(402) = 2212;.19, p < .01, but the magnitude of the correlation indicates little relationship between the use of independent and interdependent words. The use of independent and interdependent words did not vary by generational status (p > .89). Table 7 shows the correlations of interdependent and independent words with the holistic coding measures of interdependent and independent themes used in Study 1B. We also examined the correlations of the interdependent and independent linguistic word scores with the values that students selected in the VA condition and found that students used the greatest proportion of independent words in their VA essays when they selected the values: independence, learning and gaining knowledge and curiosity. Students used the greatest proportion of interdependent words in their essays when they selected the values: relationships with friends and family, belonging to a social group, and spiritual and religious values.
Table 7.
Study 1C: Percent of Students Who Selected Each Value and Correlations with LIWC Word Counts Within the Values Affirmation Condition
| Selected Value | % Who Selected Value Once or Twice |
Independent Words |
Interdependent Words |
|---|---|---|---|
| Independence | 32% | .369** | −.200** |
| Learning and Gaining Knowledge | 65% | .301** | −.120* |
| Curiosity | 10% | .151** | −.213** |
| Relationships with Friends and Family | 89% | −.252** | .531** |
| Belonging to a Social Group | 9% | −.124* | .132** |
| Spiritual and Religious Values | 21% | −.223** | .119* |
| Being Good at Art | 1% | −.037 | .064 |
| Government and Politics | 1% | −.053 | .014 |
| Athletic Ability | 9% | .078 | −.142** |
| Career | 37% | −.047 | .043 |
| Creativity | 16% | .040 | −.220** |
| Music | 10% | −.018 | −.121* |
| Nature and the Environment | 8% | −.084 | .020 |
| School Spirit | 0% | .061 | .012 |
| Sense of Humor | 37% | −.168** | −.133** |
| Social Networking and/or Gaming | 1% | −.051 | −.013 |
| Holistic Coding and Word Usage | % Who Wrote about Themes Once or Twice |
||
| Independent Themes | 36% | .458** | −.180** |
| Interdependent Themes | 92% | −.268** | .636** |
| Independent Words | -- | 1 | −.193** |
| Interdependent Words | -- | −.193** | 1 |
Note. The first 8 values listed were used in our directed conditions in Study 2.
p <.05,
p < .01.
LIWC variables as predictors of course performance
To examine the effects of independent and interdependent words on performance within VA conditions, we tested a model that included the three main effects (independent words used, interdependent words used, generational status), three 2-way interactions (Independent Words Used × Generational Status, Interdependent Words Used × Generational Status, and Independent Words × Interdependent Words), and a 3-way interaction (Independent Words × Interdependent Words × Generational Status) with gender and word count as covariates. We also tested all possible interactions between our three main effects and gender; because no gender interaction effects emerged, they were subsequently trimmed from the model.
Within the VA condition, we found a main effect of independent words such that using more independent words was associated with better performance in the class, t(392) = 2.86, p < .01, β = 0.19. This main effect was qualified by an interaction with generational status indicating that FG students, in particular, performed better in the class when they used more independent words, relative to CG students t(392) = 2.06, p = .04, β = 0.14 (see on-line supplemental materials for graph). A main effect of generational status indicated that FG students performed more poorly in the class than CG students, t(392) = 2.18, p = .03, β = −0.12 and no other main effects or interactions emerged. Interdependent words were not predictive of course performance at the main effect level (p = .99), and did not significantly interact with generational status, p > .66. These results, obtained for students in the VA condition, are consistent with the overall effects documented with holistic independence coding, tested in the whole sample (Study 1B). Considered together, our data suggest that writing about independence was especially beneficial for FG students.
Discussion
Analyzing the essays along continuums of independent and interdependent word counts revealed that using independent words was particularly beneficial for FG students’ academic performance. Conversely, analyses with interdependent word counts failed to yield any significant effects on academic performance. This pattern of results is consistent with the findings from Study 1B: writing about independence, whether coded holistically or assessed via linguistic analysis, was especially beneficial for FG students.
Studies 1B and 1C indicate that FG students who wrote about independence in their VA essays performed better in the biology course and also performed better in classes taken after the semester in which VA was implemented. They also experienced less concern about their background over the course of the semester. Thus, for FG students, the benefits of reflecting on their independence were twofold. In addition to improving both their short-term and long-term academic performance, FG students’ concern about background were alleviated suggesting that they experienced greater academic fit.
Study 2 – Laboratory Test of Independent Values Affirmation
Although the results of Studies 1B and 1C suggest that writing about independence accounted for the efficacy of the VA intervention for FG students, stronger evidence would come from an experiment in which independent writing was experimentally induced, to isolate the causal role of independent writing. In Study 2, we sought to examine the benefits of independent writing for FG students in a controlled laboratory experiment. Specifically, we manipulated the mediating variable (independence) from Study 1B, as well as interdependence, in a controlled lab setting in which students could be directed to write about either independence or interdependence in a VA essay. That is, we manipulated the extent to which participants wrote about their independence and interdependence by altering the standard VA exercise to place greater emphasis on independence or interdependence and then administered a standardized math test as a measure of academic performance. Our experimental manipulations were intended, first, to test if we could successfully induce participants to use more independent or interdependent words in their essays, and second, to examine if our experimental manipulation of students’ independent or interdependent writing would affect their performance on a math test.
Given that writing about independence mediated the performance effects for FG students documented by Harackiewicz, Cannning, et al. (2014), we predicted that FG students in the present study would perform best on the math test in conditions that elicited the most independent writing. Conversely, because we found no significant effects of interdependent writing on FG students’ performance in Studies 1B and 1C, we did not expect FG students to perform significantly better on the math test in conditions that elicited interdependent writing. For CG students, we did not expect performance on the math test to vary as a function of experimental condition.
Method
We tested two different methods for encouraging more independent or interdependent writing. In both methods, we restricted the list of values that students were given to choose from for their essays. The text analysis conducted in Study 1C allowed us to identify which values, when chosen, were most highly correlated with independent and interdependent writing (see Table 7). Thus, in the current study we limited the set of values that students could choose to write about to values associated with either independent or interdependent writing, intermixed with some neutral values. For example, in Study 1C, text analyses revealed that students wrote most about independence when they selected the values independence, learning and gaining knowledge, and/or curiosity. These text analyses also revealed that students wrote most about interdependence when they chose to write about the values of relationships with friends and family, belonging to a social group, and/or spiritual or religious values.
In the present experiment, we encouraged independent or interdependent writing by restricting the list of values from the original list of 12 values in the standard VA exercise, to just five values: the three values most highly correlated with independent (Independent VA conditions) or interdependent (Interdependent VA conditions) writing, plus two neutral values that were uncorrelated with independent and interdependent writing and rarely selected by participants in Harackiewicz, Cannning, et al. (2014): government and politics and being good at art. The inclusion of neutral values was intended to create an illusion of choice: We presented participants with a variety of values to choose from, while subtly encouraging them to write about either independence or interdependence. These conditions will be referred to as the Independent VA and Interdependent VA conditions and they represent two of the four “directed” VA conditions (see Table 8 for full description of conditions).
Table 8.
Conditions in Study 2
| Directed Conditions | ||||||
|---|---|---|---|---|---|---|
| Interdependent VA Conditions | Independent VA Conditions | |||||
| Control | Standard VA |
Interdependent VA |
Framed Interdependent VA |
Independent VA | Framed Independent VA |
|
| Values Provided | 12 values | 12 values | 5 values (3 interdependent, 2 neutral) | 5 values (3 interdependent, 2 neutral) | 5 values (3 independent, 2 neutral) | 5 values (3 independent, 2 neutral) |
| Value Instructions | Choose LEAST important | Choose MOST important | Choose MOST important | Choose MOST important | Choose MOST important | Choose MOST important |
| Writing Exercise | Write about why those values may be important to someone else | Write about why those values are important to you | Write about why those values are important to you | Write about how those values make you feel closer and more connected with other people | Write about why those values are important to you | Write about how those values make you feel independent and self-sufficient |
Note. VA = values affirmation.
In addition to restricting the values that participants could select, in the other two directed VA conditions we provided additional writing instructions. This second method of encouraging independent and interdependent writing was implemented to test whether it was necessary to add specific instructions to encourage participants to engage in more independent and interdependent writing, or whether simply constraining the values was sufficient to induce such writing. Restricting the set of values might be sufficient to encourage different types of writing, but it might be necessary to provide more explicit guidance and changing the VA instructions to create significant changes in students’ writing. We tested both possibilities.
In half of the “directed” conditions we therefore added the individuating and belonging writing prompts employed by Shnabel et al. (2013), which were successful in influencing VA writing in their study, and used an unconstrained set of values. The “individuating-affirmation” prompt asked students to write about how their most important values made them feel independent and self-sufficient, whereas the “belonging-affirmation” prompt directed students to write about why their most important values made them feel closer and more connected with other people. In the present study, these prompts were used in conjunction with the restricted set of corresponding values. That is, the individuating prompt was used with the restricted set of independent values to form the Framed Independent VA, and the belonging-affirmation prompt was employed in conjunction with the restricted set of interdependent values to create the Framed Interdependent VA. Thus, in the directed VA conditions, we used two different methods to promote independent and interdependent writing (i.e., restricted values with standard VA instructions, and restricted values with the addition of framed writing instructions) to test whether value restriction alone was sufficient or whether specific writing instructions were necessary to induce independent and interdependent writing.
In order to compare these new directed VA conditions to previous research, we also included a Standard VA condition, in which participants chose from 12 values with standard instructions and a Control condition, in which students wrote about why their least important values may be important to someone else. These conditions were identical to those used in Study 1. Based on the findings of Study 1B and 1C, we predicted that FG students would perform best in conditions that elicited the most writing about independence.
Participants
Three hundred thirty-three undergraduate students (154 males, 179 females) were recruited from an introductory psychology course and were compensated with course extra credit. Of the 333 students, 222 were CG students and 111 were FG students. Their mean age was 18.73 (SD = 1.44) with 67% of participants indicating that they were in their first year of college. According to self-report, 80% of the participants identified as White, 9% as Asian or Asian American, 6% as African American, 3% as Hispanic, 2% declined to specify a race/ethnicity, and 1 participant identified as Native American.
Procedure
We designed an experimental protocol in which generational status was made salient to students just prior to writing a VA (or control) essay, followed by a standardized math test. The experiment took place in a room designed to be reminiscent of a classroom or testing facility; there were flyers advertising standardized test preparation companies on the walls and privacy dividers between individual desks. Students were told that they would be taking a standardized test to measure their “academic ability and performance,” and they completed the experiment individually or with one or two other students of the same gender. This gendered grouping was intended to decrease the salience of gender identities, given that we were primarily interested in activating FG student identities.
Participants were told that they would be part of a study investigating students’ academic background in relation to college performance. Once participants were seated, an experimenter gave them verbal instructions and handed out forms and testing materials from a desk at the front of the room. Participants first filled out a questionnaire that asked about their academic background. Included were questions asking “Are you a first-generation college student?” and “Please mark the highest level of education your mother/father received.” In accordance with prior research, having participants designate their generational status on this questionnaire was intended to activate FG student identities (Croizet & Claire, 1998).
After completing the questionnaire, participants were given two minutes to complete a sheet of 12 double-digit multiplication problems, which was presented as a math warm-up exercise, and provided a baseline measure of math performance. The experimenter used a stopwatch to time each section of the exercises and informed participants when it was time to stop working and go on to the next task. The VA writing packet was presented next as a writing warm-up exercise. Each participant received a packet containing a list of values and writing instructions that differed according to condition. Participants were allotted 15 minutes total during which they selected 2–3 values from the list and composed their essays. The last page of the packet asked participants to summarize their thoughts in two sentences, as a final way of having them reflect on their selected values.
Next, participants completed a math test (the measure of performance), presented as the “Massachusetts Math Problem-Solving Battery,” which contained 16 GRE-style, multiple-choice math problems. Participants were given 15 minutes to solve as many problems as they could, and asked to circle their answers on a separate answer sheet.
Experimental conditions
Participants were randomly assigned to receive one of six writing packets, which differed by values list and writing instructions, depending on the condition (Table 8). Those in the Standard VA condition and the Control condition received the same VA and control essay prompts used in the Harackiewicz, Cannning, et al. (2014) study. That is, they selected 2–3 of their most or least important values from a list of 12 values and wrote about why the circled values were important to them (Standard VA) or why their 2–3 least important values might be important to someone else (Control).
The four “directed” conditions (two independent conditions and two interdependent conditions) were designed to elicit either independent or interdependent writing. The Independent VA condition restricted the number of values that participants could choose to 5 values (independence, learning and gaining knowledge, curiosity, government and politics, and being good at art) and used the standard writing instructions. The Framed Independent VA used the same list of 5 values but added a writing prompt that instructed participants to write about how the values they selected made them feel independent and self-sufficient (Shnabel et al., 2013). The two interdependent conditions were constructed in similar fashion. The Interdependent VA condition restricted the value list to 5 values (relationships with friends and family, belonging to a social group, and spiritual or religious values, government and politics and being good at art), with the standard writing instructions. The Framed Interdependent VA condition used the same list of values but utilized a writing prompt which directed participants to write about how the values they selected made them feel closer and more connected with other people (Shnabel et al., 2013).
Pre-test measures
We measured students’ motives for attending college with the abbreviated version of Stephens and colleagues’ scale, which was used in the original Harackiewicz, Cannning, et al. (2014) study. The measure directed students to indicate which of 10 items characterized important reasons for completing their college degree (checking as many as were relevant). Half the items referred to independent motives reflecting traditional American university values (e.g., “becoming an independent thinker”) and half referred to interdependent motives commonly associated with working-class values (e.g., “give back to my community;” see Table 9 for full list of motives). We also collected measures of students’ academic fit, assessed with an academic belonging scale and a belonging uncertainty scale administered to participants at the beginning of the semester. The academic belonging measure consisted of two items that had previously been used as part of a larger measure of academic belonging in Harackiewicz, Cannning, et al. (2014) study. Items were “I belong at University X,” and “I feel like an outsider in college” (reversed), α = .76. The belonging uncertainty scale validated in prior research (Walton & Cohen, 2007) included the items “When something bad happens, I feel that maybe I don’t belong at University X,” and “Sometimes I feel that I belong at University X, and sometimes I feel that I don’t belong at University X”.
Table 9.
Study 2: Percent of Interdependent and Independent Items Endorsed by First-Generation and Continuing-Generation Students
| Survey items | FG students | CG students |
|---|---|---|
| Interdependent items | ||
| Help my family out after I’m done with college** | 58% | 32% |
| Be a role model for people in my community | 51% | 40% |
| Show that people with my background can do well** | 38% | 17% |
| Give back to my community* | 54% | 42% |
| Provide a better life for my own children | 80% | 76% |
| Scale Mean** | 2.80 | 2.06 |
| Independent items | ||
| Expand my knowledge of the world* | 59% | 72% |
| Become an independent thinker | 42% | 48% |
| Explore new interests | 61% | 63% |
| Learn more about my interests | 60% | 63% |
| Expand my understanding of the world** | 48% | 64% |
| Scale mean* | 2.69 | 3.09 |
p < .05,
p < .01.
FG = first-generation; CG = continuing-generation
Results
Sample characteristics
Consistent with previous research (e.g., Harackiewicz, Cannning, et al., 2014; Stephens, Fryberg et al., 2012), FG students endorsed more interdependent motives, t(331) = 4.21, p < .001, and fewer independent motives, t(331) = 2.03, p = .04, when compared to CG students (see Table 9). At pre-test, they were also lower in academic belonging, t(322) = 1.96, p = .05, and higher in belonging uncertainty, t(322) = 2.56, p = .01, suggesting that FG students worried more about whether they belonged in college compared to CG students. Table 10 shows the means and inter-correlations of these four measures. Importantly, there were no significant gender effects or effects of condition on any of these measures, p > .19.
Table 10.
Study 2: Means and Correlations of Motives and Belonging by Generational Status
| Measure | CG students (SD) |
FG students (SD) |
1 | 2 | 3 | 4 |
|---|---|---|---|---|---|---|
| 1. Independent Motives | 3.09 (1.65) | 2.69 (1.73) | -- | |||
| 2. Interdependent Motives | 2.06 (1.51) | 2.80 (1.51) | .23** | -- | ||
| 3. Academic Belonging | 5.84 (1.26) | 5.54 (1.37) | .08 | −.02 | -- | |
| 4. Belonging Uncertainty | 3.25 (1.77) | 3.79 (1.87) | .00 | .06 | −.60** | -- |
p < .01.
FG = first-generation; CG = continuing-generation
Manipulation check
In order to test whether our directed VA conditions induced participants to write more about independence or interdependence, each essay was coded using the same holistic coding system employed in Study 1B. That is, each essay was read by at least two trained coders who evaluated whether each essay contained themes of independence and interdependence. Each essay was given two scores: a score of 0 or 1 for independence and a score of 0 or 1 for interdependence (0 = no, 1 = yes). Similar to Study 1B, our coders maintained high inter-rater reliability for both independence (Cohen’s kappa = .94) and interdependence (Cohen’s kappa = .90) coding (Landis & Koch, 1977). Initial agreement among coders was over 90% for both independence and interdependence coding and for the few instances in which coders did not agree, a third coder was consulted to resolve any ambiguity.
Logistic regression was conducted to examine the effects of each experimental condition and participants’ generational status on each dichotomous coding variable. We also examined which values were most commonly selected in our VA conditions in order to ensure that the independent and interdependent values were more frequently selected than the neutral values in our directed conditions (see on-line supplemental materials for a table of the percentage of students who selected each value by condition).
Independent themes
Logistic regression analyses revealed that using generational status and condition as predictors of writing about independence significantly improved model fit, χ2(6) = 194.32, p < .001. Generational status was not significant at the main effect level and adding in interactions between generational status and the conditions in a second step did not improve model fit, χ2(5) = 4.79, p > .44. All participants in the Independent VA and Framed Independent VA conditions were more likely to write about independence in their essays compared to participants in the Standard VA condition (p < .001). Whereas 32% of participants in the Standard VA wrote about independence, 86% of participants in the Independent VA condition, and 87% of participants in the Framed Independent VA condition wrote about independence. Conversely, the number of participants who wrote about independence in the Control condition (2%), Interdependent VA condition (4%), and the Framed Interdependent VA condition (5%) was significantly lower than in the Standard VA condition, all p < .05. As hypothesized, our Independent VA and Framed Independent VA conditions both induced participants to write significantly more about their personal independence and this effect did not vary by generational status. Importantly, the proportion of students who wrote about independence did not differ between the Independent VA and Framed Independent VA conditions, χ2(1) = .847, p > .90, indicating that both conditions elicited similar amounts of independent writing.
Interdependent themes
We tested the same model with interdependent themes as the dependent variable. Logistic regression analyses revealed that using generational status and condition as predictors of writing about interdependence significantly improved model fit, χ2(6) = 275.07, p < .001. Generational status was not significant at the main effect level, and adding interactions between generational status and the conditions in a second step did not improve model fit, χ2(5) = 10.75, p > .10. The Interdependent VA condition induced participants to write significantly more about interdependence in their essays than participants in the Standard VA condition (p = .04). Whereas 85% of participants in the Standard VA condition wrote about interdependence in their essays, 98% of participants in the Interdependent VA condition wrote about interdependence. Similarly, 95% of participants in the Framed Interdependent VA condition wrote about interdependence, but this effect did not reach statistical significance, p = .11 (this was due, in part, to the high level of interdependent writing in the Standard VA condition). Conversely, the number of participants who wrote about interdependence in the Control condition (7%), Independent VA condition (12%), and the Framed Independent VA condition (8%) was significantly lower than the number of participants in the Standard VA condition, all p < .001. As hypothesized, the Interdependent VA and Framed Interdependent VA conditions increased the likelihood that participants would write about interdependence in their essays, and this effect did not vary by generation status. Importantly, the proportion of students who wrote about interdependence did not differ between the Interdependent VA and Framed Interdependent VA conditions, χ2(1) = .469, p > .59, indicating that both conditions elicited similar amounts of interdependent writing.
It is also important to note that essays in the directed conditions often included one theme at the expense of the other. For example, the Independent VA and Framed Independent VA conditions induced participants to write more about independence, but significantly less about interdependence when compared to the Standard VA condition. Similarly, the Interdependent VA and Framed Interdependent VA conditions led participants to write more about interdependence but significantly less about independence compared to participants in the Standard VA condition. This allowed us to examine the performance of students who wrote primarily about independence (and very little about interdependence) and compare it to those who wrote primarily about interdependence (and less about independence).
The manipulation check also allowed us to examine if simply restricting the set of possible values participants could write about was sufficient, or if was necessary to prompt participants to elaborate on how the values made them feel more independent or connected to others to encourage independent or interdependent writing, respectively. The number of participants who wrote about independence did not significantly vary across the two independent conditions (86% in the Independent VA condition and 87% in the Framed Independent VA condition). Similarly, the number of participants who wrote about interdependence did not significantly vary across the two interdependent conditions (98% in the Interdependent VA condition and 95% in the Framed Interdependent VA condition). Therefore, we combined the two independent conditions and the two interdependent conditions for analyses on math test performance, resulting in a 4-cell design (Independent Combined VA, Interdependent Combined VA, Standard VA and Control).11
Math Test Performance
We used a set of three orthogonal contrasts to test the effects of the four conditions on math test performance. The first contrast tested all three VA conditions against the Control condition (VA vs. control). The second contrast compared the two directed VA conditions with the Standard VA condition (Directed vs. Standard VA). A third contrast compared the Independent Combined VA (+1) condition to the Interdependent Combined VA (−1) condition (Independent vs. Interdependent VA). The three contrasts were entered into a multiple regression model along with the main effects of generational status (CG = −1, FG = 1), gender (female = −1, male = 1), and the performance covariate.12 All possible interactions between generational status, the three contrasts, and gender were tested. None of the gender interaction terms were significant, and were therefore trimmed from the model, reducing the final model to 9 terms: the main effects of the three contrasts, generational status, gender, and the pre-test performance covariate, as well as three 2-way interactions between the contrasts and generational status.
There were three significant main effects on math test performance. The main effect of generational status revealed that FG students (M = 8.59, SD = 2.94) performed more poorly than CG students (M = 9.50, SD = 2.58) on the math test, t(323) = 3.27, p < .001, β = −0.17. A main effect of gender showed that females (M = 8.54, SD = 2.65) performed more poorly than males (M = 9.97, SD = 2.64), t(323) = 4.27, p < .001, β = 0.26. Finally, the pre-test performance covariate was a significant predictor of math test performance, t(323) = 4.27, p < .001, β = 0.22.
In addition to these three main effects, there was a significant 2-way interaction between the Independent vs Interdependent VA contrast and generational status, indicating that FG students solved significantly more math problems in the Independent VA conditions (M = 9.62, SD = 2.59), compared to the Interdependent Conditions (M = 8.00, SD = 3.07), t(323) = 2.22, p = .03, β = 0.12 (see Figure 5). The Independent vs Interdependent VA contrast effectively tests the significance of the hypothesized linear relationship between independence and performance within VA conditions, given that the three VA conditions are coded according to how much writing about independence they induced: Interdependent Combined VA (−1), Standard VA (0), Independent Combined VA (+1). This linear pattern for FG students as a function of the degree of independent writing prompted is depicted in Figure 5.13 No other effects were significant.
Figure 5.
Study 2: Mean number of correct answers and +/−1 standard errors for continuing-generation and first-generation participants as a function of affirmation condition.
In order to examine which, if any, of the VA conditions were effective relative to the Control condition, we also analyzed math test performance with dummy codes, using the Control condition as the reference group. We used the same model previously reported on Math Test Performance but with three dummy codes in place of planned orthogonal contrasts, and found that in addition to the main effects reported above, there was a main effect of the Independent Combined Condition relative to control t(323) = 2.07, p = .04, β = 0.14. In other words, all participants performed better in the Independent Combined Condition, relative to control. However, as predicted, this effect was moderated by generational status such that FG students in particular performed especially well in the Independent Combined Condition, t(323) = 2.36, p = .02, β = 0.18. No other significant effects emerged. As can be seen in Figure 5, FG students performed more poorly than CG students in the control condition (the social class gap). However, in the Independent VA conditions, the gap was completely closed and, in fact, FG students scored slightly better than CG students.
Gender effects
Given that many previous studies have used VA to reduce identity threat for women on math tasks, we also tested for gender differences and interactions, but found no VA effects for women (all p > .15). This is unsurprising for several reasons. First, the present study was intentionally designed to evoke FG student identities by asking FG students to denote their generational status at the start of the experiment. Indeed, previous studies have shown that making a threatened identity salient in an evaluative setting has the potential to activate specific social identity threat (Croizet & Claire, 1998; Martens, Johns, Greenberg, & Schimel, 2006; Steele & Aronson, 1995). In contrast to making FG status more salient, gender was made less salient by conducting the experiment in same-gendered groupings in which gender and gender stereotypes were never mentioned. Second, women in this sample did not differ from men on any of the characteristics that contribute to the cultural mismatch FG students contend with. Whereas FG students in the present study reported fewer independent motives, more interdependent movies, less academic belonging, and more belonging uncertainty than their CG counterparts, women did not respond differently than men on any of these measures, all p > .19. This suggests that the cultural mismatch that afflicts FG students may not be analogous to any kind of mismatch that women taking a math test might experience.
Discussion
The results of Study 2 provide further support for our hypothesis that reflecting on independence can improve performance for FG students, and the experimental design allows us to make a stronger inference about causality. For example, in Study 1B, we were able to rule out and control for some potentially confounding variables in the mediation analyses, but there may have been an unmeasured third variable that was associated with both independent writing and academic performance for FG students. In this experimental study however, we were able to show that writing about independence had a causal effect on performance.
Consistent with the literature on cultural mismatch, the FG students in our sample reported more interdependent motives, fewer independent motives, and more uncertainty about their academic fit relative to CG students. Generally, FG students with these characteristics have not performed as well as their CG peers, and we observed this social class performance gap in the control condition of the experiment. However, when FG students were encouraged to reflect on their personal independence, they performed just as well as CG students. This suggests that encouraging FG students to reflect on the personal value of independence may be a particularly effective strategy for promoting their academic success.
Furthermore, given that the two independent conditions elicited similar amounts of independent writing, and that the two interdependent conditions elicited similar amounts of interdependent writing, we can conclude that altering the standard VA prompts did not produce any significant increases in students’ writing about independent themes, relative to simply restricting the values list. Restricting the values that students could choose from to those most highly correlated with independent and interdependent writing in previous research was sufficient to increase the amount of independent and interdependent writing in students’ essays.
Consistent with the results of Study 1B indicating that the positive VA effect for FG students in the original field study (Harackiewicz, Canning et al. 2014) was strongest for FG students who affirmed their independence, Study 2 revealed that it was only when students were led to write about independence that they incurred the benefits of VA. The fact that this pattern of results emerged across two different samples, using different measures of academic performance, supports our hypothesis that affirming independence can be particularly beneficial for FG students.
General Discussion
The present research advances our understanding of values affirmation and the potential for improving the performance of FG students with VA interventions. The results of Study 1 indicate that, consistent with prior research (Cohen et al., 2009; Sherman et al., 2013), VA interventions can induce long-term benefits for students’ academic performance. Given that the post-intervention GPA of FG students in the VA condition was, on average, nearly one fifth of a GPA point higher than FG students in the control condition (Study 1A), it appears that VA has the potential to change the academic trajectory of FG students.
In addition to documenting the power and durability of VA effects, the present research also shed light on the mechanisms driving positive VA effects for FG students. In both a randomized field study and a controlled laboratory experiment, we demonstrated that FG students performed better on academic tasks when they wrote about independence in their VA essays. Study 1B demonstrated that writing about independence accounted for the positive effect of VA on FG students’ grades as well as the mitigating effect of VA on FG students’ concerns about their academic background. Study 2 demonstrated that varying the extent to which FG students wrote about independence had a significant effect on their performance on a math test.
Experimental manipulations that elicited more independent writing produced better math performance for FG students. Taken together, these results suggest that affirming independence can positively impact the academic performance of FG students, which has important implications for our understanding of the mechanisms by which values-affirmation interventions can close achievement gaps.
Implications for Cultural Mismatch Theory
Why is it beneficial for FG students, a group noted for their more interdependent values and motives for attending college, to write about independence? Research often points to a lack of academic belonging as a reason for the lagging academic performance of FG students (e.g., Johnson et al., 2011; Ostrove & Long, 2007), and cultural mismatch theory has posited that this perceived lack of academic fit may reflect a sense of incongruence between FG students’ interdependent motivations for attending college and traditional university norms of independence. Prior work by Stephens and colleagues (Stephens, Fryberg, et al., 2012; Stephens, Townsend, et al., 2012) has sought to address cultural mismatch by changing perceptions of the university. For example, when FG students were presented with a welcome letter that portrayed the university’s values as more interdependent (matching their own motivational orientations), they experienced a challenging task (giving a speech) as less stressful (Stephens, Townsend, et al., 2012) and performed better on anagram and tangram tasks (Stephens, Fryberg, et al., 2012). In Study 2, we promoted cultural match in a different way, by emphasizing the similarities between FG students’ values and university culture. Rather than changing students’ perceptions of the university or of themselves, the independent VA intervention simply asks students to reflect on personal values which are also consistent with the university norms. These findings corroborate previous research documenting how fostering a perceived cultural match can be beneficial for FG students (Stephens, Fryberg, et al., 2012; Stephens, Townsend, et al., 2012), and highlight the potential for leveraging FG students’ independent values for promoting cultural match.
It is important to note that even though FG students generally endorse more interdependent motives for attending college, this does not preclude them from also being independent. Independent and interdependent values are not mutually exclusive: FG students do espouse independent motives, and universities do value some aspects of interdependence (e.g., teamwork, collaboration; Stephens, Fryberg, et al., 2012). Consistent with previous reports of student motives, FG students in Study 2 reported fewer independent motives than their CG counterparts, but 86% of FG students still selected at least one independent motive as being an important reason for obtaining a degree. In fact, FG and CG students were equally likely to endorse 3 of the 5 independent motives (see Table 9). Our work suggests that an overemphasis on the differences between FG student values and university norms in the cultural mismatch literature may overlook powerful interventions that can leverage similarities between personal values and university norms.
Thus, the independent values affirmation was not designed to make FG students more independent; rather, it was designed to encourage them to reflect on their existing independent values. The independent values used in the VA intervention in Study 2 (independence, learning and gaining knowledge, and curiosity) represent values that are all implicated in the pursuit of a college degree. Because they are striving to be the first in their family to obtain a college degree, it follows that FG students may value independence, learning and gaining knowledge, and curiosity to some extent. Providing FG students with the opportunity to affirm their independent values may help them find the common ground between their own personal values and the values of the university. Increasing the salience of areas of match between personal values and university culture should improve perceived academic fit and reduce academic concerns, and the results of Study 1B support this: FG students who affirmed their independence were less concerned about their background at the end of the semester.
These results are also consistent with research on identity-based motivation which has shown that students are often more motivated, engaged, and able to perform better when they perceive an activity as being congruent with their identity (Oyserman & Destin, 2010). By reflecting on their own independence, FG students may have activated an adaptive identity for functioning in the independent culture that characterizes higher education. One could even argue that affirming independence is akin to affirming the “academic self.” Thus, it follows that when FG students affirm their own independence, they are reflecting on the part of their identity that is consistent with the values implicit in their academic context.
Another intervention that has proven successful in addressing the social class achievement gap utilized a related strategy, emphasizing the positive aspects of FG students’ backgrounds. A difference-education intervention that highlighted college students’ diverse backgrounds and provided incoming first-year FG students with narratives of successful FG student experiences led to better college adjustment and improved grades for FG students (Stephens, Hamedani & Destin, 2014; Stephens, Townsend, Hamedani, Destin, & Manzo, 2015). By emphasizing the success stories of FG students, the difference-education intervention signaled that that the university culture was not catered exclusively for students from middle class (and more independent) backgrounds. In this way, FG students’ interdependent motives for attending college may have been experienced as more consistent with the university context. This suggests that there are multiple ways of intervening to help FG students and that increasing their sense of academic fit may be paramount to their college success.
In sum, recent work indicates that fostering a perceived match between FG students’ values and the culture of higher education has the potential to help FG students. In both Stephens and colleagues’ work and the present research, experimental manipulations encouraged students to perceive a match between aspects of themselves and the university environment, resulting in improved academic outcomes for FG students. A logical next question is what is a more effective and feasible way of addressing this mismatch? Should administrators strive to transform their university culture to be more consistent with FG student motivations (i.e., place more value on interdependence) or should educators encourage FG students to reflect on the importance of their own independent values? Given that emerging research suggests that creating a cultural match (either by changing the perception of the university context, or by encouraging students to reflect on their independent values) has the potential to help FG students, future research should consider both possibilities.
Implications for Values Affirmation Research
Prior research has identified mechanisms that can be important in VA interventions (e.g., construal level, social belonging). With the current studies, we ask the important question of whether the same mechanisms should be expected to work for different populations. Our results are consistent with some of the proposed mechanisms of VA interventions, but also implicate some processes novel to this area of research.
One mechanism through which VA helps threatened students perform better is by broadening the perspective through which students assess salient threats (i.e., promoting higher levels of construal), which buffers against ego depletion (Cohen & Sherman, 2014; Schmeichel & Vohs, 2009). Students confronted with an identity threat expend valuable cognitive resources attending to the threat and tend to construe negative events in more concrete terms (i.e., at a lower level of construal). Because their attention is narrowly focused on the threat, they lack the potential to perceive threat through a broader perspective (i.e., at a higher level of construal) that is more adaptive for long-term goals such as earning a college degree (Wakslak & Trope, 2009). Values affirmation has been shown to promote high-level construal (Wakslak & Trope, 2009) and previous research has noted that higher-level construal leads to greater self-control over cognitive resources (Vallacher & Wegner, 1987) and has the potential to counteract ego depletion (Schmeichel & Vohs, 2009). For example, Sherman et al. (2013) found that in addition to improved academic performance, affirmed Latino American students reported higher levels of construal and experienced the adversity in school as less related to their identity than Latino American students in the control condition.
Although we did not measure construal level, it is possible that VA promoted higher levels of construal among FG students (particularly for those who wrote about independence) in the present studies and future research should explore this possibility. When some FG students arrive on campus, the contrast between the college context and their own cultural background may be overwhelming, causing them to doubt their place in higher education, and creating the type of threat associated with lower levels of construal. By providing these students with the opportunity to affirm their personal values, we may have helped FG students confront potential identity threats from a broader perspective, thereby alleviating the adversity attributed to identity threat and promoting academic fit and performance.
Another mechanism shown to be important for the positive effects of the VA intervention is social belonging. Shnabel and colleagues (2013) identified social belonging (i.e., writing that reminds students of their interdependence) as the mechanism driving the positive effects of VA for Black middle-school students. However, in the current research, writing about interdependence failed to yield positive effects for FG students in both the field study (Study 1) and the laboratory study (Study 2). It is not the case that writing about interdependence had negative effects, per se. In fact, more than 90% of FG students in the field study who wrote about their independence also wrote about interdependence. However, it does appear that writing about interdependence to the exclusion of independence precluded FG students from fully benefiting from the VA interventions in these studies. Although social belonging has been documented as a “key ingredient” for middle-school students to write about in order to produce positive VA effects (Shnabel et al., 2013), our findings that independence is a “key ingredient” for first-generation college students warrants further investigation. Furthermore, Cook, Purdie-Vaughns, Garcia, and Cohen (2011) have noted that values affirmation affects middle schoolers’ sense of academic belonging (a composite measure of students’ sense of social belonging and potential to succeed in school) suggesting that values-affirmation research should continue to examine intervention effects on the various components of belonging (e.g., social belonging, academic belonging or fit, belonging uncertainty, concern about background).
Our research suggests that the mechanisms underlying VA effects may differ depending on which population the intervention is intended to help. For example, for underrepresented minority middle-school students, problems of adjustment may be inherently social in nature and thus for these students, it may be more beneficial to reflect on valued social connections. For FG college students, on the other hand, the problems of belonging may have more to do with their academic background and sense of academic fit. It is important to note, however, that interdependence or social belonging may be more important for some FG students, given that interdependence may be particularly important for FG students from Hispanic, Native American, or Asian backgrounds, for instance (Covarrubias, Herrmann, & Fryberg, 2016; Smith, Cech, Metz, Huntoon, & Moyer, 2014; Hoshino-Browne, Zanna, Spencer, Zanna, Kitayama, & Lackenbauer, 2005; Thoman, Brown, Mason, Harmsen, & Smith, 2015). The present studies were conducted with samples of FG students who were 91% (Harackiewicz, Canning et al., 2014) and 83% (Study 2) majority students (i.e., Caucasian or Asian), whereas previous research on FG students has documented that as many as half of FG students may be underrepresented minorities (e.g., Stephens, Fryberg, et al., 2012).
Researchers can tailor interventions to the characteristics of the populations they intend to help and the specific problems that need to be addressed (Harackiewicz, Canning Tibbetts, Priniski & Hyde, in press; Harackiewicz, Tibbetts, Canning, & Hyde, 2014; Walton, 2014). It is important to consider the motivational characteristics of students that interventions are designed to help, and the present findings suggest that it is possible to design interventions to better suit the populations we intend to serve. Whereas reflecting on social belonging may be important for middle-school students, it may be more important for FG students to reflect on values that promote a sense of academic fit. In fact, recent research suggests that models of academic belonging should extend beyond social belonging and incorporate students’ perceptions of their academic abilities and intellectual fit (Lewis & Hodges, 2015). These less social (and more academic) components of belonging may be critically important for FG students and the present work suggests that affirming independence may be one way to increase FG students’ sense of academic belonging and ultimately improve their academic performance.
Conclusion
Although FG students may struggle in college in comparison to their CG peers, the present findings suggest that social-psychological interventions can be effective in promoting their academic performance. Building on previous research that has demonstrated the positive effects of values affirmation (e.g., Cohen et al., 2006) and the advantage of perceiving a cultural match between students’ motivations and university values (Stephens, Fryberg, et al., 2012), the present studies indicate that the social-class achievement gap can be addressed by designing VA interventions to target the unique challenges faced by FG students. By focusing on independence in a values-affirmation intervention, we can help FG students leverage their own independent values to overcome the social identity threats brought on by cultural mismatch.
Supplementary Material
Acknowledgments
This research was supported by National Institutes of Health Grant R01GM102703 and the Institute of Education Sciences, U.S. Department of Education, through Award R305B090009 to the University of Wisconsin–Madison. Elizabeth Canning was supported by the National Science Foundation Program under Grant No. DGE-1256259. The opinions expressed are those of the authors and do not represent views of the U.S. Department of Education, the National Science Foundation, or the National Institutes of Health.
We thank Kerstin Krautbauer, Cindee Giffen, and Kerry Martin for their management of this project, as well as course coordinators Julie Collins, Jean Heitz, Brian Parks, Carlos Peralta, and Jon Breschak, and all the research assistants who helped to implement both the field and laboratory studies. We thank Seth Blair and Douglas Rouse, co-chairs of the Introductory Biology course for their support of the project and we thank Markus Brauer and Brian Allen for their help.
Footnotes
Extending these analyses into a fourth semester would cause us to lose an additional 60 students (45 students graduated after spring, 2013, 8 dropped or transferred and 7 were not enrolled in fall, 2013) from the original Harackiewicz, Canning et al. (2014) sample.
The number of students who graduated, took 2 or more semesters off, or had dropped/transferred by spring 2013 did not differ as a function of generational status, or condition, p > .19.
Excluding the 17 students we imputed does not change the pattern or significance of the results.
The number of students who graduated or were still enrolled in classes did not differ as a function of generational status or condition (p > .30). However, there was an effect of generational status on the percentage of students who dropped or transferred after the intervention semester, χ2(1) = 6.38, p = .021, indicating that a greater proportion of FG students dropped or transferred (12%) than CG students (6%); this effect did not differ as a function of treatment condition. The ten students who did not take any classes after the fall 2011 semester did not differ as a function of generational status, χ2(1) = 2.79, p > .10, or VA condition, χ2(1) = 1.68, p > .21.
Consistent with the methods of Harackiewicz, Canning et al. (2014), we repeated analyses assessing long-term performance (e.g., semester grades, post-intervention GPA), controlling for academic performance prior to the intervention (by using a standardized composite measure of students’ prior semester GPA, ACT, and SAT scores). As in the original Harackiewicz, Canning et al. (2014) study, including this performance covariate did not change the pattern or significance of the results.
Shnabel’s conceptualization of social belonging is consistent with our conceptualization of interdependence.
In the Harackiewicz, Canning et al. (2014) original analysis a significant main effect of lecture section on biology course grade, t(789) = 3.11, p < .01, β = .14, indicated that students in lecture 1 obtained higher grades than students in the other two lectures. There was also a significant generational status × lecture interaction on biology grade suggesting that there was a larger social class achievement gap in lecture 2, t(789) = 2.18, p = .03, β = −.10. Thus, we controlled for lecture section in analyses of outcome measures taken from the semester in which VA was implemented and found that no other effects of lecture section were significant on biology course grade or concern about background, indicating that the treatment effects did not vary as a function of lecture section.
There were too few underrepresented minority FG students (18 total; 10 in control, 8 in VA) to examine whether the effect of VA and independent writing for FG students also varied as a function of minority status. However, the pattern of means suggests that, consistent with our findings for all FG students, underrepresented minority FG students benefited from writing about independence.
Because there is no existing protocol for using a sensitivity analysis to test for possible confounding variables in moderated mediation models we conducted a sensitivity analysis exclusively among FG students to examine potential confounds in models testing independence as a mediator of treatment effects (i.e., we conducted a sensitivity analysis on simple unmoderated mediation models that included VA as the independent variable, writing about independence as the mediator, and course grades/post-intervention GPA as the dependent variable). The average causal mediation effect among FG students was significant on course grade, 95% CI: = [.026, .081], p = .02, and post-intervention GPA 95% CI: = [.011, .052], p = .04. Furthermore, the results indicated that an unobserved confounder could account for up to 9%–13% (course grade) and 8%–10% (post-intervention GPA) of the variance in both the dependent variable and the mediator before the average causal mediation effect would become zero. Thus, although we had to use oversimplified sensitivity analysis models to test for possible confounders, these results suggest that the mediation findings are fairly robust to possible unobserved pre-treatment confounders.
Adding the effects of ACT to regression models testing the basic treatment effects does not change the significance or interpretation of the reported treatment effects for FG students.
The performance of CG and FG students did not differ between the two independent-oriented VA conditions (Independent VA and Framed Independent VA) or between the two interdependent-oriented VA conditions (Interdependent VA and Framed Interdependent VA), p > .23.
The math warm-up exercise was included as a measure of baseline math ability and thus served as a performance covariate in all models that include math test performance as a dependent variable; the pattern and significance of results remained consistent regardless of whether or not it was included in the model.
Replacing the orthogonal contrasts with polynomial contrasts (linear, quadratic, and cubic) ordered according to how much writing about independence each condition induced (setting the Control group lowest, −3, Interdependent Combined VA at −1, Standard VA at +1, and Independent Combined VA at +3), revealed conceptually similar results. A main effect of the linear contrast indicated that on average, students performed better in conditions in which there was more independent writing, t(323) = 2.29, p = .02, β = 0.13. However, consistent with our hypothesis, this main effect was qualified by an interaction with generational status indicating that this linear pattern was especially strong for FG students such that they performed better in conditions that induced more writing about independence, t(323) = 2.43, p = .02, β = 0.13.
References
- Autor DH. Skills, education, and the rise of earnings inequality among the “other 99 percent”. Science. 2014;344:843–851. doi: 10.1126/science.1251868. [DOI] [PubMed] [Google Scholar]
- Betts JR, Morell D. The determinants of undergraduate grade point average: The relative importance of family background, high school resources, and peer group effects. The Journal of Human Resources. 1999;34:268–293. [Google Scholar]
- Bullock JG, Green DP, Ha SE. Yes, but what’s the mechanism?(don’t expect an easy answer) Journal of personality and social psychology. 2010;98(4):550. doi: 10.1037/a0018933. [DOI] [PubMed] [Google Scholar]
- Chen X. U.S. Department of Education, National Center for Education Statistics. Washington, DC: U.S. Government Printing Office; 2005. First-generation students in postsecondary education: A look at their college transcripts. Retrieved from http://nces.ed.gov/pubs2005/2005171.pdf. [Google Scholar]
- Cohen GL, Garcia J, Apfel N, Master A. Reducing the racial achievement gap: a social-psychological intervention. Science. 2006;313:1307–1310. doi: 10.1126/science.1128317. [DOI] [PubMed] [Google Scholar]
- Cohen GL, Garcia J, Purdie-Vaughns V, Apfel N, Brzustoski P. Recursive processes in self-affirmation: Intervening to close the minority achievement gap. Science. 2009;324:400–403. doi: 10.1126/science.1170769. [DOI] [PubMed] [Google Scholar]
- Cohen GL, Sherman DK. The psychology of change: Self-affirmation and social psychological intervention. Annual Review of Psychology. 2014;65:333–371. doi: 10.1146/annurev-psych-010213-115137. [DOI] [PubMed] [Google Scholar]
- Cook JE, Purdie-Vaughns V, Garcia J, Cohen GL. Chronic threat and contingent belonging: Protective benefits of values affirmation on identity development. Journal of Personality and Social Psychology. 2012;102:479–496. doi: 10.1037/a0026312. [DOI] [PubMed] [Google Scholar]
- Covarrubias R, Herrman SD, Fryberg SA. Affirming the interdependent self: Implications for Latino student performance. Basic and Applied Social Psychology. 2016;38:47–57. [Google Scholar]
- Croizet JC, Claire T. Extending the concept of stereotype threat to social class: The intellectual underperformance of students from low socioeconomic backgrounds. Personality and Social Psychology Bulletin. 1998;24:588–594. [Google Scholar]
- Geiser S, Santelices MV. Validity of high-school grades in predicting student success beyond the freshman year: High-school record vs. standardized tests as indicators of four-year college outcomes. University of California, Berkeley Center for Studies in Higher Education Research & Occasional Paper Series: CSHE.6.07. 2007 [Google Scholar]
- Grove WA, Wasserman T. The life-cycle pattern of collegiate GPA: Longitudinal cohort analysis and grade inflation. The Journal of Economic Education. 2004;35:162–174. [Google Scholar]
- Hanselman P, Bruch SK, Gamoran A, Borman GD. Threat in context: School moderation of the impact of social identity threat on racial/ethnic achievement gaps. Sociology of Education. 2014;87:106–124. [Google Scholar]
- Harackiewicz JM, Canning EA, Tibbetts Y, Giffen CJ, Blair SS, Rouse DI, Hyde JS. Closing the social class achievement gap for first-generation students in undergraduate biology. Journal of Educational Psychology. 2014;106:375–389. doi: 10.1037/a0034679. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Harackiewicz JM, Canning EA, Tibbetts Y, Priniski SJ, Hyde JS. Closing achievement gaps with a utility-value intervention: Disentangling race and social class. Journal of Personality and Social Psychology. doi: 10.1037/pspp0000075. (in press) [DOI] [PMC free article] [PubMed] [Google Scholar]
- Harackiewicz JM, Tibbetts Y, Canning EA, Hyde JS. Harnessing values to promote motivation in education. In: Karabenick SA, Urden TC, editors. Advances in motivation and achievement. Vol. 18. Bingley, UK: Emerald Group Publishing Limited; 2014. pp. 71–105. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Hayes AF. Introduction to mediation, moderation, and conditional process analysis: A regression based approach. New York, NY: The Guilford Press; 2013. [Google Scholar]
- Hoshino-Browne E, Zanna AS, Spencer SJ, Zanna MP, Kitayama S, Lackenbauer S. On the cultural guises of cognitive dissonance: the case of easterners and westerners. Journal of personality and social psychology. 2005;89(3):294. doi: 10.1037/0022-3514.89.3.294. [DOI] [PubMed] [Google Scholar]
- Imai K, Yamamoto T. Identification and sensitivity analysis for multiple causal mechanisms: Revisiting evidence from framing experiments. Political Analysis. 2013;21:141–171. [Google Scholar]
- Jackman MR, Jackman RW. Class awareness in the United States. Berkeley: University of California Press; 1983. [Google Scholar]
- Johnson SE, Richeson JA, Finkel EJ. Middle class and marginal? Socioeconomic status, stigma, and self-regulation at an elite university. Journal of Personality and Social Psychology. 2011;100:838–852. doi: 10.1037/a0021956. [DOI] [PubMed] [Google Scholar]
- Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics. 1977;33:159–174. [PubMed] [Google Scholar]
- Lewis KL, Hodges SD. Expanding the concept of belonging in academic domains: Development and validation of the Ability Uncertainty Scale. Learning and Individual Differences. 2015;202:197–202. [Google Scholar]
- Madera JM, Hebl MR, Martin RC. Gender and letters of recommendation for academia: Agentic and communal differences. Journal of Applied Psychology. 2009;94:1591–1599. doi: 10.1037/a0016539. [DOI] [PubMed] [Google Scholar]
- Martens A, Johns M, Greenberg J, Schimel J. Combating stereotype threat: The effect of self-affirmation on women’s intellectual performance. Journal of Experimental Social Psychology. 2006;42:236–243. [Google Scholar]
- Markus HR, Kitayama S. Culture and the self: Implications for cognition, emotion, and motivation. Psychological Review. 1991;98:224–253. [Google Scholar]
- McQueen A, Klein WMP. Experimental manipulations of self-affirmation: A systematic review. Self and Identity. 2006;5:289–354. [Google Scholar]
- Miyake A, Kost-Smith LE, Finkelstein ND, Pollock SJ, Cohen GL, Ito TA. Reducing the gender achievement gap in college science: A classroom study of values affirmation. Science. 2010;330:1234–1237. doi: 10.1126/science.1195996. [DOI] [PubMed] [Google Scholar]
- Ostrove JM, Long SM. Social class and belonging: Implications for college adjustment. The Review of Higher Education. 2007;30:363–389. [Google Scholar]
- Oyserman D, Destin M. Identity-based motivation: Implications for intervention. The Counseling Psychologist. 2010;38:1001–1043. doi: 10.1177/0011000010374775. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Pascarella ET, Terenzini PT. How college affects students: Findings and insights from twenty years of research. San Francisco, CA: Jossey-Bass; 1991. [Google Scholar]
- Pennebaker JW, Francis ME, Booth RJ. Linguistic inquiry and word count: LIWC 2001. Mahwah, NJ: Erlbaum Publishers; 2001. [Google Scholar]
- Reardon SF. The widening academic achievement gap between the rich and the poor: New evidence and possible explanations. In: Murnane R, Duncan G, editors. Whither opportunity? Rising inequality and the uncertain life chances of low-income children. New York, NY: Russell Sage Foundation Press; 2011. pp. 91–116. [Google Scholar]
- Saenz VB, Hurtado S, Barrera D, Wolf D, Yeung F. Los Angeles, CA: Higher Education Research Institute; 2007. First in my family: A profile of first-generation college students at four-year institutions since 1971. Retrieved from http://www.heri.ucla.edu/PDFs/pubs/TFS/Special/Monographs/FirstInMyFamily.pdf. [Google Scholar]
- Schmeichel BJ, Vohs KD. Self-affirmation and self-control: Affirming core values counteracts ego depletion. Journal of Personality and Social Psychology. 2009;96:770–782. doi: 10.1037/a0014635. [DOI] [PubMed] [Google Scholar]
- Sherman DK, Hartson KA, Binning KR, Purdie-Vaughns V, Garcia J, Taborsky-Barba S, Cohen G. Deflecting the trajectory and changing the narrative: How self-affirmation affects academic performance and motivation under identity threat. Journal of Personality and Social Psychology. 2013;104:591–618. doi: 10.1037/a0031495. [DOI] [PubMed] [Google Scholar]
- Shnabel N, Purdie-Vaughns V, Cook JE, Garcia J, Cohen GL. Demystifying values-affirmation interventions: Writing about social belonging is a key to buffering against identity threat. Personality and Social Psychology Bulletin. 2013;39:663–676. doi: 10.1177/0146167213480816. [DOI] [PubMed] [Google Scholar]
- Sirin SR. Socioeconomic status and academic achievement: A meta-analytic review of research. Review of Educational Research. 2005;75:417–453. [Google Scholar]
- Smeding A, Darnon C, Souchal C, Toczek-Capelle M-C, Butera F. Reducing the socio-economic status achievement gap at university by promoting mastery-oriented assessment. PLoS ONE. 2013;8:e71678. doi: 10.1371/journal.pone.0071678. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Smith ER. Attitudes and Social Cognition - editorial. Journal of Personality and Social Psychology. 2012;102(1):1. doi: 10.1037/a0026676. [DOI] [PubMed] [Google Scholar]
- Smith JL, Cech E, Metz A, Huntoon M, Moyer C. Giving back or giving up: Native American student experiences in science and engineering. Cultural Diversity and Ethnic Minority Psychology. 2014;20:413–429. doi: 10.1037/a0036945. [DOI] [PubMed] [Google Scholar]
- Snibbe AC, Markus HR. You can’t always get what you want: educational attainment, agency, and choice. Journal of Personality and Social Psychology. 2005;88:703–720. doi: 10.1037/0022-3514.88.4.703. [DOI] [PubMed] [Google Scholar]
- Steele CM. A threat in the air: How stereotypes shape intellectual identity and performance. American Psychologist. 1997;52:613–629. doi: 10.1037//0003-066x.52.6.613. [DOI] [PubMed] [Google Scholar]
- Steele CM, Aronson J. Stereotype threat and the intellectual test performance of African Americans. Journal of personality and social psychology. 1995;69:797. doi: 10.1037//0022-3514.69.5.797. [DOI] [PubMed] [Google Scholar]
- Stephens NM, Fryberg SA, Markus HR, Johnson CS, Covarrubias R. Unseen disadvantage: How American universities' focus on independence undermines the academic performance of first-generation college students. Journal of Personality and Social Psychology. 2012;102:1178. doi: 10.1037/a0027143. [DOI] [PubMed] [Google Scholar]
- Stephens NM, Hamedani MG, Destin M. Closing the social-class achievement gap: A difference-education intervention improves first-generation students’ academic performance and all students’ college transition. Psychological Science. 2014;25:943–953. doi: 10.1177/0956797613518349. [DOI] [PubMed] [Google Scholar]
- Stephens NM, Townsend SSM, Markus HR, Phillips LT. A cultural mismatch: Independent cultural norms produce greater increases in cortisol and more negative emotions among first-generation college students. Journal of Experimental Social Psychology. 2012;48:1389–1393. [Google Scholar]
- Stephens NM, Townsend SSM, Hamedani MG, Destin M, Manzo V. A difference-education intervention equips first-generation college students to thrive in the face of stressful college situations. Psychological Science. 2015;10:1556–1566. doi: 10.1177/0956797615593501. [DOI] [PubMed] [Google Scholar]
- Terenzini PT, Springer L, Yaeger PM, Pascarella ET, Nora A. First-generation college students: Characteristics, experiences, and cognitive development. Research in Higher Education. 1996;37:1–22. [Google Scholar]
- Thoman DB, Brown ER, Mason AZ, Harmsen AG, Smith JL. The role of altruistic values in motivating underrepresented minority students for biomedicine. BioScience. 2015;65:183–188. doi: 10.1093/biosci/biu199. [DOI] [PMC free article] [PubMed] [Google Scholar]
- University of Wisconsin-Madison Office of the Registrar. Average Semester Undergraduate Grade Point Averages [Data file] 2015 Retrieved from https://registrar.wisc.edu/undergraduate_grade_point_averages.htm. [Google Scholar]
- Vallacher RR, Wegner DM. What do people think they're doing? Action identification and human behavior. Psychological Review. 1987;94:3–15. [Google Scholar]
- Wakslak CJ, Trope Y. Cognitive consequences of affirming the self: The relationship between self-affirmation and object construal. Journal of Experimental Social Psychology. 2009;45(4):927–932. doi: 10.1016/j.jesp.2009.05.002. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Walton GM. The new science of wise interventions. Current Directions in Psychological Science. 2014;23:73–82. doi: 10.1177/0963721420904453. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Walton GM, Cohen GL. A question of belonging: Race, social fit, and achievement. Journal of Personality and Social Psychology. 2007;92:82–96. doi: 10.1037/0022-3514.92.1.82. [DOI] [PubMed] [Google Scholar]
- Warburton EC, Bugarin R, Nunez AM. Washington, DC: National Center for Education Statistics, U.S. Government Printing Office; 2001. Bridging the gap: Academic preparation and postsecondary success of first-generation students (NCES 2001-153) Retrieved from http://nces.ed.gov/pubs2001/2001153.pdf. [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.