Promoting Persistence in the Biological and Medical Sciences: An Expectancy-Value Approach to Intervention

Abstract

A wide range of occupations require science, technology, engineering, and mathematics (STEM) skills, yet almost half of students who intend to pursue a post-secondary STEM education abandon these plans before graduating from college. This attrition is especially pronounced among underrepresented groups (i.e., racial/ethnic minorities and first-generation college students). We conducted a two-year follow-up of a utility-value intervention that had been implemented in an introductory biology course. This intervention was previously shown to improve performance in the course, on average and especially among underrepresented students, reducing the achievement gap. The goal of the present study was to examine whether the intervention also impacted persistence in the biomedical track throughout college. The intervention had a more positive impact on long-term persistence for students who were more confident that they could succeed at the beginning of the course, and this effect was partially driven by the extent to which students reflected on the personal relevance of biological topics in their essays. This mechanism was distinct from the process that had been found to underlie intervention effects on performance – engagement with course material – suggesting that utility-value interventions may affect different academic outcomes by initiating distinct psychological processes. Although we did not find that the intervention was differentially effective for underrepresented students in terms of persistence, we found that positive effects on performance were associated with increased persistence for these students. Results suggest that utility-value interventions in an introductory course can be an effective strategy to promote persistence in the biomedical sciences throughout college.

Educational Impact and Implications Statement

We conducted a two-year follow-up study of a utility-value intervention (UVI) in which students wrote about the personal value of course topics in an introductory biology course for biomedical majors, and examined persistence in terms of subsequent course-taking and whether students were biomedical majors. The original study found that the UVI improved course grades for all students, on average, as well as for underrepresented students, by promoting engagement with intervention writing assignments. In the current follow-up, we found that the UVI indirectly increased persistence through the original effects on course grades, for all students on average and for underrepresented students in particular: better course grades were associated with higher levels of persistence. In addition, we found that among students who received the UVI, those who were more confident that they could succeed in the course were more likely to persist in a biomedical education two years later, an effect explained by increased focus on personal relevance in students’ biology writing assignments. This study has important implications for college science educators who seek to retain their students and promote diversity in the biomedical sciences.

Across a wide variety of occupations, there is an increasing demand for professionals with strong skills in the areas of science, technology, engineering, and mathematics (STEM; National Science Board, 2014). However, almost half (48%) of bachelor’s degree students who initially declare a major in a STEM field leave the STEM disciplines before completing college, thereby limiting their opportunities to develop these STEM skills (Chen, 2013). Furthermore, the STEM workforce lacks diversity. Students from disadvantaged groups are less likely to obtain postsecondary STEM degrees than their majority peers. Underrepresented racial/ethnic minorities (URMs) account for 26% of the population in the United States, yet they receive only 11% of college STEM degrees (National Science Board, 2014). First-generation (FG) college students (i.e., students for whom neither parent has a four-year college degree) are also less likely to receive STEM degrees, dropping out of college at a much higher rate (28–35%) than their continuing-generation (CG) peers (17%; Radford, Berkner, Wheeless, & Shepherd, 2010). Given the importance of increasing the number of professionals in the STEM workforce and improving the representation of disadvantaged groups in these disciplines, it is crucial to test strategies to promote persistence in STEM fields. Psychologists can address these problems by designing interventions based in theories from social and developmental psychology (Harackiewicz & Priniski, 2018).

In higher education, students’ decisions regarding whether to leave (or persist in) STEM often occur soon after taking a single introductory course, because this early experience shapes expectations for future STEM courses. College students who remain in a STEM field after an introductory course are more likely to later obtain a degree in that field (Chen, 2013). Therefore, by promoting students’ motivation and performance in an introductory course, an intervention may encourage students to persist beyond the course and set them on a positive trajectory toward obtaining a STEM degree. We investigate this possibility in the present study.

One way to improve motivation and performance in college courses is by emphasizing the utility value (i.e., usefulness) of course material. This approach is based in Eccles’ expectancy-value theory of academic choices (Eccles & Wigfield, 2002), which posits that students’ academic decisions (e.g., which courses to take, which major to pursue) are determined by their expectancies for success and subjective task values (i.e., the perceived value or relevance of a task, including its utility value). To the degree that students perceive value in a field and believe that they can succeed, they are more likely to choose to pursue an education in that field. Furthermore, students’ expectancies for success and subjective task values are theorized to positively interact to predict their academic choices: students are most likely to pursue an education in a particular domain if they both value the domain and believe they can succeed in it. Longitudinal studies have found that expectancies for success and subjective task values are correlated with interest and engagement in academic courses, as well as the courses, majors, and careers that students choose to pursue (Harackiewicz, Durik, Barron, Linnenbrink-Garcia, & Tauer, 2008; Jacobs, Lanza, Osgood, Eccles, & Wigfield, 2002; Wang, 2012), and that effects are strongest among students with both high expectancies for success and high perceptions of task value (Nagengast et al., 2011; Guo et al., 2016). Expectancy-value theory predicts that an intervention that increases subjective task value in a particular domain should promote pursuit of that domain, particularly among students who are confident that they can succeed. These confident students already believe they can pursue the domain, but an intervention that increases subjective task value may help them to see why pursuing that domain is worthwhile.

Experimental studies show that students who receive a utility-value intervention (UVI), in which they complete writing assignments describing how academic material is relevant or useful, develop more interest and perform better (Canning & Harackiewicz, 2015; Hulleman, Godes, Hendricks, & Harackiewicz, 2010). Recent field studies have demonstrated that this intervention can improve performance in high school and college STEM courses, particularly among students who have a history of poor academic performance or doubt their academic ability, indicating a negative interaction between the UVI and expectancies for success (Harackiewicz, Canning, Tibbetts, Priniski, & Hyde, 2016; Hulleman & Harackiewicz, 2009; Hulleman, Kosovich, Barron, & Daniel, 2017). These findings are counterintuitive. In contrast to the positive expectancy-value interaction theorized with respect to prospective academic decisions (e.g., choosing courses and majors), the negative expectancy-value interaction observed on performance (a proximal academic outcome) indicates a compensatory effect whereby reflecting on the value of course topics helps the students who are most likely to struggle in the course. Prior research suggests that generating examples of utility may help lower-performing students to become more interested in the material, which can promote learning (Hulleman & Harackiewicz, 2009; Hulleman et al., 2010). This compensatory process may be driven by increased engagement with the course material, which leads to improved understanding and better performance.

Indeed, research has provided support for an engagement-driven effect of the UVI on academic performance. Harackiewicz et al. (2016) conducted a large-scale randomized controlled trial testing the effectiveness of a UVI for closing achievement gaps in an introductory college biology course. They examined the intersection of race and social class (defined in terms of FG status) and tested whether the UVI was particularly effective for URM (i.e., African American, Latino/a, and Native American) students, FG college students, and/or “FG-URM” students (who are both FG and URM; Cole, 2009). These groups face distinct sets of challenges in college and tend to underperform compared to their CG-majority peers (Fiske & Markus, 2012; Lee, 2002). Therefore, students who are both FG and URM may be at a distinct disadvantage (Jack, 2014). Harackiewicz et al. (2016) found that the UVI improved course grades, averaged across all students, and this effect was especially strong for students with a history of poor academic performance. Moreover, the UVI was particularly effective for FG-URM students, improving their course grades by 0.51 grade points and closing the achievement gap relative to their CG-Majority peers by 61%. Students’ engagement in the writing assignments (as indicated by essay length) mediated these effects, providing evidence for one process by which the UVI improves the course performance of students who are most likely to struggle: by fostering engagement with the material.

Although the UVI has been found to improve performance in introductory STEM courses, less is known about whether this intervention affects long-term decisions to pursue a STEM education. Could the UVI also encourage college students to take additional STEM courses and even pursue a major in a STEM field? Canning and colleagues (2018) recently tested the effects of a UVI on college students’ persistence in STEM shortly after an introductory course. They found that students who received the intervention in an introductory biology course obtained higher grades in the course, were less likely to abandon their intended STEM major by the end of the semester, and were more likely to take another biology course the following semester. These findings provide the first evidence that a UVI may be able to increase short-term persistence in STEM, in addition to course performance, and suggest that one route by which the UVI could promote STEM degree attainment is by increasing the likelihood that students will enroll in another STEM course (see also Rosenzweig et al., 2018). However, the study is limited in that persistence was measured only one semester after the intervention, leaving questions about longer-term impacts unanswered. Therefore, in the present research, we examined whether a UVI can influence academic decisions over a longer time course.

On the one hand, it may seem unlikely that a series of brief writing assignments focused on the relevance and usefulness of specific topics within a single course could have a long-term impact on students’ STEM persistence. However, there are at least two reasons to hypothesize that the benefits of the UVI would be detectable a year or more later. First, we know that the UVI can improve students’ grades in gateway introductory courses, and these improved grades may have downstream consequences. Higher grades in introductory courses may be a prerequisite for continuing on to higher-level courses, and may also influence students’ beliefs. A higher grade in a difficult introductory course can signal that students are on the right track and competent enough to continue on in the major, whereas a lower grade may lead students to conclude that they can’t make it, and consider changing their major to another field. Thus, the UVI may influence STEM persistence indirectly, by improving grades in introductory courses. Because the effects of the UVI on course grades are well established, this seems the most likely process through which long-term effects of the UVI may be detected. However, it is also possible that the UVI could have direct effects on long-term persistence. Expectancy-value theory predicts that students will be most likely to pursue a STEM field if they value the field and expect to be successful (Eccles & Wigfield, 2002). To the extent that the UVI helps students to focus on the value of STEM topics, it should increase STEM persistence, with the strongest effects for those who are confident that they can succeed. In sum, the UVI may influence long-term persistence directly, through expectancy-value processes, or indirectly, through higher grades in introductory courses.

When examining the potential long-term effects of an intervention, it is important to consider the motivational processes by which the intervention may operate in the long term and whether they would be different from the processes that operate in the short term (Harackiewicz & Priniski, 2018). Whereas engagement with course material may promote course performance, we would not expect such a task-specific process to generalize beyond that particular course. In contrast, a more general sense of the personal relevance of science may encourage students to continue their education within the domain, especially among students who are already confident that they could do so successfully (Priniski, Hecht, & Harackiewicz, 2018). In other words, UVIs may initiate two motivational processes: task engagement and personal relevance, with different implications for different outcomes (performance in the short term vs. academic decisions in the longer term) and different populations (e.g., students who struggle vs. confident students). Task engagement may be especially important for students who struggle in the class, because it helps them learn more effectively, contributing to positive effects on performance, whereas personal relevance may more strongly influence the academic decisions of confident students.

Linguistic analysis of students’ writing assignments may allow us to identify and test these two motivational processes (Beigman Klebanov, Burstein, Harackiewicz, Priniski, & Mulholland, 2017). Harackiewicz et al. (2016) documented the process by which the UVI promotes performance through engagement with course material by counting the number of words used in the assigned essays (i.e., essay length). They also determined the degree to which students were able to make specific connections between the material and their own lives with human coding (Harackiewicz et al., 2016; Canning et al., 2018). However, more sophisticated techniques can provide insight into the ways in which students think about course topics more broadly. Indeed, research suggests that the words individuals use can capture thought processes that serve as powerful indicators of important outcomes, such as trauma recovery, mental and physical health, and academic success (Pennebaker, 2011; Pennebaker, Chung, Frazee, Lavergne, & Beaver, 2014). Linguistic Inquiry and Word Count (LIWC) software, for example, can be used to assess the degree to which students reflect on scientific concepts with regard to themselves or others by detecting the frequency of particular pronouns (Pennebaker, Booth, & Frances, 2007). This type of linguistic analysis may be particularly informative because it goes beyond the specific topic connections that students make in an introductory science course and provides a more general indicator of the degree to which students think about science in personal terms. Such an indicator might allow researchers to assess the personal relevance process initiated by UVIs and could be key for understanding longer-term effects on STEM persistence.

Current Study

The goal of the present study is to examine two critical issues: the potential of a UVI to affect long-term pursuit of an education in STEM, and the mechanisms through which this may occur. A two-year follow up of the original Harackiewicz et al. (2016) study was conducted to address the following questions: (a) can the UVI promote persistence in a STEM field two years after implementation in an introductory course; and if so, (b) are long-term effects stronger for particular groups of students (e.g., FG-URM students, confident students), and (c) are long-term effects driven by a different motivational process than the engagement process documented for the short-term effects on performance?

This research examined long-term effects of the UVI on persistence within the biological and medical sciences (hereafter biomedical sciences). Retaining students in the biomedical sciences is particularly important because an influx of diverse professionals into these fields would help fuel innovation in medical research and treatment (Finkelstein & Hambrick, 1996). However, attrition is especially pernicious in the biomedical sciences, as students are 23% less likely to remain in these fields throughout college compared to mathematics, engineering, and the physical sciences (Bettinger, 2010). The Harackiewicz et al. (2016) study was conducted in an introductory biology course taken by prospective majors and pre-health students. This study was extended by examining whether the intervention produced positive effects on persistence in the biomedical track. Specifically, we investigated effects on persistence soon after the biology course (i.e., enrollment in the next course in the biology sequence) and two years later (i.e., majoring in a biomedical field two years post-intervention).

Method

Participants

The UVI was implemented in the first course of a two-course introductory biology sequence in a double-blind randomized controlled trial at a large Midwestern university. This sequence is one option for fulfilling introductory biology requirements at the university. It serves as a primary pathway to premedical preparation and post-secondary study in the biological sciences because it is designed specifically for biomedical sciences students, and the first course is a prerequisite for the second. A total of 2,378 students (8% URM, 21% FG) were enrolled in the course over the four semesters in which the intervention was conducted. All consenting FG and URM students were included in the study, along with a randomly selected subset of CG-majority students (82% White, 18% Asian or Asian American). Of the 1,060 students eligible for the study, 1,039 completed the course and gave consent for access to their academic records (15 students dropped the course, and 6 students did not consent). The sample for the present study is comprised of these 1,039 students (60% women), of whom 422 were CG-majority, 427 were FG-majority, 126 were CG-URM, and 64 were FG-URM.¹ 579 of these students were assigned to the UVI condition and 460 were assigned to the control condition. Of these 1,039 students, 918 (88%) took the course in their second year of college.

Procedure

This follow-up study tracked students over the two years following the semester in which they received the UVI. Students had been randomly assigned within lecture sections to the UVI condition or the control condition, stratified by gender, FG, and URM status. Instructional staff in the course were all blind to experimental condition. In both conditions, students were assigned to complete three writing assignments throughout the course of the semester, presented as graded course assignments. Students in the control and UVI conditions were both asked to:

Select a concept or issue that was covered in lecture and formulate a question. Select the relevant information from class notes and the textbook, and write a 1–2 page essay.

In the control condition, students were asked to answer their question by summarizing material from the course:

Select the relevant information from class notes and the textbook, and write a one to two-page response to your question. You should attempt to organize the material in a meaningful way, rather than simply listing the main facts or research findings. Remember to summarize the material in your own words.

Writing assignments in the UVI condition varied somewhat throughout the study, but in each assignment, students were asked to address their question using material from the course and write about the relevance of the topic. In an essay variation, students were asked to connect the course material to their own life:

Write an essay addressing this question and discuss the relevance of the concept or issue to your own life. Be sure to include some concrete information that was covered in this unit, explaining why this specific information is relevant to your life or useful for you. Be sure to explain how the information applies to you personally and give examples.

In a letter variation of the UVI, students were asked to connect the material to other individuals:

Write a letter to a family member or close friend, addressing this question, and discuss the relevance of this specific concept or issue to this other person. Be sure to include some concrete information that was covered in this unit, explaining why the information is relevant to this person’s life or useful for this person. Be sure to explain how the information applies to this person and give examples.

Biology graduate students graded the assignment for scientific accuracy and adherence to instructions. The assignments were cumulatively worth 1.8% of the grade in the course (0.6% per assignment). This study was approved by the Institutional Review Board.

Measures

Measures were collected at five time points: (1) during the first week of the introductory biology course, prior to administering the intervention (FG and URM status, prior GPA, confidence about performance in the course, baseline major), (2) during the course (linguistic measures computed from the three writing assignments: engagement and personal focus), (3) at the end of the course (course grade), (4) the next semester (enrollment in the second introductory biology course), and 5) two years post-intervention (persistence in a biomedical major; see Figure 1). Zero-order correlations between measures are presented in Table 1.

Figure 1.

Timeline of measures taken throughout the study.

Table 1.

Zero-Order Correlations

	1	2	3	4	5	6	7
1. Confidence about performance	—
2. Prior GPA	.13***	—
3. Course grade	.23***	.58***	—
4. Personal focus	.00	−.01	−.01	—
5. Engagement	.06*	.21***	.27***	.02	—
6. Enrollment in second biology course	.10**	.13***	.27***	.01	.07*	—
7. Biomedical major	.09**	.17***	.31***	−.05	.08*	.58***	—

^***p < .001.

^**p < .01.

^*p < .05.

Baseline measures.

In a questionnaire administered at the beginning of the course, students reported the highest level of education achieved by their parents. Students with at least one parent who had obtained a bachelor’s degree or higher were classified as CG (continuing-generation), and all other students were classified as FG (first-generation). Students also reported their race/ethnicity; those who self-identified as African American, Latino/a or Native American were classified as URM (underrepresented minority), and all other students were classified as majority. Three questionnaire items (“I am confident that I will do well in Introductory Biology;” “I expect to get a good grade in this course;” “I am confident that I can obtain a final grade of B or better in this course”) were combined to create the measure of confidence about performance (α = .82). Students were also asked to report their intended or current major. Students were coded as being in a biomedical major at baseline if (a) their intended major was in a biomedical field, or (b) they indicated that they were on a pre-health educational track (i.e., pre-medical, pre-dental, pre-pharmacy, pre-veterinary, pre-optometry). According to these criteria, 972 students (94% of the sample) were categorized as being in a biomedical major at baseline. Coding of academic majors for all students in the sample is provided in Table S1.

Prior GPA and course grades.

Students’ grade point averages (GPAs) from previous semesters were obtained from institutional records.² Final grades in the course were provided by course instructors at the end of the semester. Instructors calculated grades from students’ performance in lecture, discussion, and laboratory sections including multiple-choice and short-answer exams and quizzes (60%), laboratory activities (35%), and discussion activities (5%). The present sample comprises eight lecture sections across four semesters, and course coordinators worked to ensure standardization of content and grading procedures across sections and semesters. Grades were assigned on a 4.0 scale: A = 90%−100% (4.0 grade points), AB = 88%−89.99% (3.5 grade points), B = 80%−87.99% (3.0 grade points), BC = 78%−79.99% (2.5 grade points), C = 70%−77.99% (2.0 grade points), D = 60%−69.99% (1.0 grade points) and F = <60% (0 grade points). Course grades were not curved.

Engagement and personal relevance.

We used two linguistic measures to assess the processes initiated by the UVI: engagement and personal relevance. We used Harackiewicz et al.’s (2016) measure of engagement with the writing assignments by averaging the number of words used across the three writing assignments. In terms of personal relevance, prior work has used either a binary complier index, which assesses whether or not students made a utility value connection in each assignment, as a manipulation check (Hulleman & Cordray, 2009), or a measure of articulated utility value, which assesses the quality of the specific utility-value connections made in the assignments (Harackiewicz et al., 2016; Canning et al., 2018); these two measures were highly correlated (r = .81) in this sample. However, because students in UVI conditions were explicitly asked to write about utility value, both the binary complier and articulated utility value measures are best conceptualized as manipulation checks.

For the current study, we examined a linguistic measure of personal relevance, because such measures capture differences in the way students write, rather than the content of their writing. Specifically, we developed a measure of “personal focus,” or the degree to which students were thinking and writing about biology in terms of its relation to their own or others’ lives. We measured personal focus by assessing the extent to which students wrote about science in personal terms by examining the number of first-person singular and second-person pronouns (e.g., I, my, you, your) as a proportion of total words in students’ essays using LIWC (Pennebaker et al., 2007). As a proportional measure, personal focus indicates the degree to which students’ biology writing assignments were saturated with personal content. Whereas the binary complier index was designed to maximally distinguish between UVI and control writing assignments, and articulated utility value was designed to measure the quality of connections in the sentences in which students discussed utility value, the personal focus measure goes beyond specific utility-value connections to provide a more general indicator of the degree to which students wrote about science in personal terms, across the entire essay.³ The measures of engagement and personal focus were independent (r = .02).

Enrollment in the second biology course.

Students intending to major in the biomedical sciences typically take two or three introductory-level biology courses, and students who take the course in which the UVI was implemented typically take the second course in that sequence (often the next semester). Thus, enrollment in the second course is a good indicator of progression toward timely completion of a biomedical degree. We obtained enrollment data from institutional records. A total of 842 students (81% of the sample) enrolled in the second biology course; of these students, 91% did so the semester immediately following the first course.

Persistence in a biomedical major.

Students’ majors were obtained from institutional records four semesters post-intervention. We measured persistence in a biomedical major at this point in time because most students would be in their senior year of college or would have graduated. Indeed, 85% of the sample were seniors or had graduated by the time of this follow-up, and 10% were juniors. A small proportion of the sample (3%) was identified as having left the institution (e.g., transferred, dropped out)⁴, and the remaining 1% were sophomores.

Majors were coded as biomedical if the student’s major was in a biomedical field or if the student was still on a pre-health educational track (e.g., a pre-medical student with a psychology major).⁵ A total of 793 students (76%) were categorized as being in a biomedical major two years post-intervention: 743 (88%) of the 842 students who took the second introductory biology course and 50 (25%) of the 197 students who did not take the second course (see Table 2).

Table 2.

Frequencies of Enrollment in the Second Biology Course x Biomedical Major

		Enrollment in second biology course
		No	Yes
Persistence in biomedical major	No	147 (14%)	99 (9%)
	Yes	50 (5%)	743 (72%)

Results

Analysis Plan

Analyses were designed to test the effects of the UVI on persistence in a biomedical field two years post-intervention, and the motivational processes through which the intervention has its effects. We examined whether (and for whom) the UVI had direct effects on measures of persistence (enrollment in the second biology course (or not) and biomedical major (or not), two years post-intervention). We also tested whether the intervention indirectly affected persistence, via the more proximal effects on course performance reported by Harackiewicz et al. (2016). Finally, we examined the processes by which the UVI might promote long-term persistence. Specifically, we tested whether linguistic indicators of engagement and personal focus would mediate any long-term effects of the intervention.

Path models using maximum likelihood estimation with a logit function were used to examine effects on the binary persistence outcomes: enrollment in the second biology course (enrollment = 1, no enrollment = 0) and biomedical major (biomedical major = 1, non-biomedical major = 0) two years post-intervention.⁶ Coefficients for effects on these binary outcomes represent log-odds. Predicted probabilities are displayed graphically to aid interpretation. Additional details about our analytic approach and model fit can be found in the Supplemental Materials.

Direct Intervention Effects on Persistence

The effects of the UVI were tested on two measures of persistence (enrollment in the second biology course and biomedical major) using the same set of independent variables that Harackiewicz et al. (2016) used to predict course grade: UVI (UVI = 1, control = −1), FG status (FG = 1, CG = −1), URM status (URM = 1, majority = −1), all two-way interactions and the three-way interaction between these variables, prior GPA (standardized), confidence about performance (standardized), and the two two-way interactions between the UVI and each of these variables. Additionally, baseline major (biomedical = 1, non-biomedical = −1) was included as a covariate to control for baseline differences in majoring intentions; this was the basic model for all analyses.

The main effect of the UVI was not significant for either enrollment in the second course or biomedical major two years post-intervention, in contrast to the significant main effect on course grade originally observed in this study (Harackiewicz et al., 2016), and in contrast to the significant main effect on enrollment in a second course in a different sample (Canning et al., 2018). However, there was a significant UVI x confidence interaction on both enrollment in the second biology course (z = 2.08, b = 0.18, p = .04) and biomedical major two years post-intervention (z = 2.62, b = 0.21, p = .01; see Figure 2). This pattern of effects suggests that the UVI moderated the degree to which confidence influenced students’ persistence in the biomedical track. To probe this possibility, we tested simple effects of confidence for students in the control and UVI conditions. In the control condition, confidence about performance did not significantly affect enrollment (z = −.34, b = −.04, p = .74) or major (z = −1.17, b = −.14, p = .24), but in the UVI condition, there was a significant positive effect of confidence on enrollment (z = 2.75, b = .31, p < .01) and major (z = 2.67, b = .28, p < .01). These findings suggest that confidence played an important role in decisions about whether to persist in the biomedical track for students who were asked to reflect on the value of course material, but not for students who were not prompted to engage in such reflection. We further probed this interaction by testing simple effects of the UVI at high and low levels of confidence (+/− 1 SD). The effects of the UVI were more positive for students with higher levels of confidence (z = 1.74, b = .25, p = .08 for enrollment; z = 1.43, b = .19, p = .15 for major) than for students with lower levels of confidence (z = −.80, b = −.11, p = .42 for enrollment; z = −1.73, b = −.23, p = .08 for major), but none of these simple effects were significant.

Figure 2.

UVI x confidence about performance effect on the probability of enrolling in the second biology course (panel A) and persisting in a biomedical major (panel B). There was no effect of confidence on enrollment (p = .74) or major (p = .24) for students in the control condition, but for students who received the UVI, confidence positively predicted both enrollment (p < .01) and major (p < .01).

Students with higher prior GPAs and students who entered the course intending to pursue a biomedical major were more likely to enroll in the second course (ps < .001). URM students were less likely to enroll in the second biology course than majority students (p = .01). This pattern of effects also emerged on biomedical major, except that the URM effect was not statistically significant (p = .17). Neither FG nor URM status significantly interacted with the UVI to predict enrollment or major, suggesting that the UVI did not increase persistence differentially for FG or URM students (ps > .25; see Table 3 for full model output).

Table 3.

Effects from the Model Testing Direct Intervention Effects on Persistence in the Biomedical Track

	b	z	p
Enrollment in second course
UVI	0.07	0.64	0.52
FG	−0.01	−0.09	0.93
URM	−0.27	−2.55	0.01
UVI x FG	−0.11	−0.99	0.32
UVI x URM	−0.02	−0.21	0.83
FG x URM	0.02	0.15	0.88
UVI x FG x URM	−0.03	−0.28	0.78
Prior GPA	0.30	3.59	0.00
UVI x Prior GPA	0.00	0.00	1.00
Confidence about performance	0.13	1.57	0.12
UVI x Confidence about performance	0.18	2.08	0.04
Baseline major	1.19	8.40	0.00
Biomedical major
UVI	−0.02	−0.18	0.86
FG	0.12	1.16	0.25
URM	−0.14	−1.37	0.17
UVI x FG	−0.06	−0.52	0.60
UVI x URM	0.00	−0.03	0.98
FG x URM	0.12	1.19	0.23
UVI x FG x URM	−0.12	−1.12	0.26
Prior GPA	0.41	5.13	0.00
UVI x Prior GPA	0.03	0.39	0.69
Confidence about performance	0.07	0.83	0.41
UVI x Confidence about performance	0.21	2.62	0.01
Baseline major	1.21	8.13	0.00

Note: Coefficients represent log-odds.

We also examined whether these two significant intervention effects were sequential, such that promoting enrollment in the next course for confident students led to long-term persistence in a biomedical major for confident students. We tested a second path model adding enrollment in the second course as a predictor of biomedical major. The pattern of effects was identical to those found in the previous analysis (see Table 4), but we also found a strong effect of enrollment in the second course on biomedical major (z = 13.96, b = 2.88, p < .001). Students who enrolled in the second course were 58.22% more likely to major in biomedical fields than students who did not enroll in the second course. Indeed, we found that the UVI x confidence effect on biomedical major was partially mediated by enrollment in the second course (z = 2.06, p = .04), such that within the UVI condition, students with higher confidence were more likely to enroll in the second course, and enrollment in turn in turn increased the probability of a persisting in a biomedical major two years post-intervention.

Table 4.

Effects from the Model Testing Enrollment in the Second Course as a Mediator of the Intervention Effect on Persistence in a Biomedical Major

	b	z	p
Enrollment in second course
Effects on enrollment are identical to those in Table 3.
Biomedical major
UVI	−0.06	−0.49	0.62
FG	0.17	1.36	0.17
URM	0.00	−0.02	0.98
UVI x FG	0.00	−0.02	0.99
UVI x URM	0.01	0.11	0.92
FG x URM	0.16	1.32	0.19
UVI x FG x URM	−0.14	−1.09	0.28
Prior GPA	0.37	3.89	0.00
UVI x Prior GPA	0.05	0.55	0.58
Confidence about performance	0.01	0.09	0.93
UVI x Confidence about performance	0.19	1.98	0.048
Enrollment in second course	2.88	13.96	0.00
Baseline major	0.80	4.46	0.00

Note: Coefficients represent log-odds.

The finding that the UVI moderated the effects of confidence on long-term persistence is consistent with the theorized positive expectancy-value interaction, which posits that students must both perceive value perceive value and expect to succeed to be motivated to pursue a particular academic path. In the UVI condition, students were encouraged to reflect on the value of course content three times across the semester, and it was in this condition that more confident students were significantly more likely to persist, which is consistent with a synergistic expectancy x value effect. As reported in Harackiewicz et al. (2016), there were no significant effects of FG or URM status on baseline confidence, suggesting that this UVI x confidence effect pertained to all students, regardless of FG and URM status. We did not find any direct effects of the UVI for FG-URM students.

Testing Linguistic Mediators of Intervention Effects

We next examined linguistic indicators of (a) personal focus and (b) engagement as possible mediators of the UVI x confidence effect on persistence in the biomedical track. We reasoned that the UVI should lead students to write about science content in a more personal way (i.e., with a more personal focus), which in turn might predict persistence in the biomedical track for more confident students. We also tested our measure of engagement (i.e., essay length) as a potential mediator, to determine whether the same process that explained short-term intervention effects on course performance in the Harackiewicz et al. (2016) study might also play a role in the long-term intervention effects on persistence.

Separate moderated mediation models were estimated for each of these possible mediators, testing the basic model on each mediator, and then testing the effect of the mediator and the mediator x moderator (confidence) interaction on enrollment. The personal focus model included a main effect of the UVI on personal focus (z = 28.09, b = 0.77, p < .001), as well as a significant personal focus x confidence interaction (z = 2.43, b = 0.33, p = .02) indicating that personal focus had a more positive effect on enrollment in the second course for students who were more confident (see Figure 3 and Table 5). Indeed, personal focus partially mediated the UVI x confidence effects on enrollment (z = 2.42, p = .02) and the UVI x confidence effect on biomedical major, through enrollment (z = 2.39, p = .02). All other effects were consistent with those reported from the basic model, with the exception that the UVI x confidence effects on enrollment and biomedical major became weaker, as would be expected with a significant moderated mediation model. Engagement did not mediate the direct effects on persistence (ps > .13).⁷

Figure 3.

Panel A displays the conceptual moderated mediation model of UVI effects on persistence in the biomedical track via personal focus, using an oval to depict the moderator variable (i.e., confidence) and a rhombus to depict the linguistic process variable (i.e., personal focus). Panel B displays the effect of confidence on enrollment in the second biology course at high (+.75 SD) and low (−.75 SD) levels of personal focus.

Table 5.

Effects from the Model Including Personal Focus as a Mediator of Intervention Effects on Persistence in the Biomedical Track

	b	z	p
Personal focus
UVI	0.77	28.09	0.00
FG	0.01	0.21	0.83
URM	0.03	1.32	0.19
UVI x FG	0.04	1.25	0.21
UVI x URM	0.02	0.70	0.49
FG x URM	0.02	0.66	0.51
UVI x FG x URM	0.04	1.47	0.14
Prior GPA	0.00	−0.21	0.84
UVI x Prior GPA	0.01	0.64	0.52
Confidence about performance	0.00	0.10	0.92
UVI x Confidence about performance	0.01	0.38	0.70
Baseline major	0.01	0.69	0.49
Enrollment in second course
UVI	0.06	0.40	0.69
FG	0.01	0.05	0.96
URM	−0.28	−2.65	0.01
UVI x FG	−0.09	−0.82	0.41
UVI x URM	−0.02	−0.21	0.84
FG x URM	0.02	0.16	0.87
UVI x FG x URM	−0.03	−0.28	0.78
Prior GPA	0.30	3.64	0.00
UVI x Prior GPA	0.01	0.12	0.91
Confidence about performance	0.16	1.86	0.06
UVI x Confidence about performance	−0.08	−0.62	0.54
Personal focus	0.03	0.18	0.86
Personal focus x Confidence about performance	0.33	2.43	0.02
Baseline major	1.20	8.47	0.00
Biomedical major
Effects on major are identical to those in Table 4.

Note: Coefficients represent log-odds for effects on enrollment and major. All other coefficients are standardized.

The analyses thus far suggest that personal focus is the more important mediator for understanding long-term effects of the UVI, but is this process independent from the process by which the UVI promotes short-term performance through engagement, or might personal focus also play a role in intervention effects on performance? To investigate this question, we tested the same moderated mediation model used in the original Harackiewicz et al. (2016) study but also included personal focus as a potential mediator. Consistent with the original findings, we found that engagement mediated the main effect of the UVI (z = 2.38, p = .02) and the UVI x FG x URM effect (z = 2.56, p = .01) on course grade, but personal focus was not a significant mediator of either of these effects (ps > .13).⁸

Taken together, these findings suggest that the UVI can affect two distinct outcomes – short-term performance and long-term academic decisions – by influencing two different motivational processes. The UVI x confidence interaction on long-term persistence was mediated by personal focus, suggesting that the UVI prompted students to write about science in a more personal way, which in turn led more confident students to remain in the biomedical track. Conversely, the finding that the effect of the UVI on course grade (overall, and particularly for FG-URMs) was mediated by engagement suggests that to the extent that these students engaged more with specific material from the course, the UVI helped them to improve their performance in that course.

The Role of Course Performance in Long-term Persistence

Whereas the UVI did not differentially affect persistence in the biomedical track for FG-URM students, it is possible that the UVI effect on course grade for these students might promote performance indirectly. Therefore, we next assessed the degree to which improved grades predicted long-term persistence, controlling for the direct effects of the intervention on persistence found in our prior analyses (see Figure 4 for a conceptual diagram). We started with our previous model and added biology course grade (as predicted by the basic model and engagement) as a predictor of enrollment in the second course and biomedical major. Grade in the first course had a strong effect on both measures of persistence: a change of one standard deviation in grade corresponded, on average, to a 10.24% increase in the predicted probability of enrolling in the second course (z = 6.71, b = 0.73, p < .001) and an 8.44% increase in the predicted probability of enrolling in a biomedical major (z = 5.08, b = 0.61, p < .001; see Table 6).

Figure 4.

Conceptual model of UVI effects on persistence in the biomedical track via personal focus, engagement and course grade, using ovals to depict moderators (i.e., confidence about performance, FG and URM status) and rhombi to depict linguistic process variables (i.e., personal focus, engagement). The dashed arrow represents indirect effects of the UVI on long-term persistence via engagement and course grade.

Table 6.

Direct Effects from the Model Including Personal Focus, Engagement, and Course Grade as Potential Mechanisms of UVI Effects on Persistence in the Biomedical Track

	b	z	p
Personal focus
Effects on personal focus are identical to those in Table 5.
Engagement
UVI	0.11	2.66	0.01
FG	0.02	0.47	0.64
URM	−0.04	−1.37	0.17
UVI x FG	0.05	1.09	0.27
UVI x URM	0.06	1.39	0.17
FG x URM	0.00	0.05	0.96
UVI x FG x URM	0.12	2.91	0.00
Prior GPA	0.20	6.15	0.00
UVI x Prior GPA	0.01	0.17	0.87
Confidence about performance	0.04	1.19	0.23
UVI x Confidence about performance	0.01	0.33	0.75
Baseline major	0.00	0.07	0.95
Course grade
UVI	0.06	1.84	0.07
FG	−0.03	−0.96	0.34
URM	−0.13	−4.97	0.00
UVI x FG	0.00	0.08	0.93
UVI x URM	0.05	1.58	0.11
FG x URM	0.00	0.13	0.90
UVI x FG x URM	0.07	2.01	0.045
Prior GPA	0.53	20.49	0.00
UVI x Prior GPA	−0.06	−2.24	0.03
Confidence about performance	0.14	5.71	0.00
UVI x Confidence about performance	0.03	1.28	0.20
Engagement	0.13	5.44	0.00
Baseline major	0.03	1.06	0.29
Enrollment in second course
UVI	−0.02	−0.12	0.91
FG	0.03	0.23	0.82
URM	−0.16	−1.42	0.16
UVI x FG	−0.11	−0.99	0.32
UVI x URM	−0.06	−0.58	0.56
FG x URM	0.01	0.11	0.92
UVI x FG x URM	−0.10	−0.89	0.37
Prior GPA	−0.09	−0.86	0.39
UVI x Prior GPA	0.06	0.69	0.49
Confidence about performance	0.06	0.61	0.54
UVI x Confidence about performance	−0.07	−0.49	0.63
Personal focus	0.08	0.53	0.60
Personal focus x Confidence about performance	0.29	1.98	0.048
Course grade	0.73	6.71	0.00
Baseline major	1.24	8.38	0.00
Biomedical major
UVI	−0.10	−0.75	0.45
FG	0.21	1.62	0.11
URM	0.10	0.79	0.43
UVI x FG	0.00	−0.02	0.98
UVI x URM	−0.02	−0.18	0.86
FG x URM	0.18	1.37	0.17
UVI x FG x URM	−0.18	−1.37	0.17
Prior GPA	0.05	0.41	0.68
UVI x Prior GPA	0.10	0.97	0.33
Confidence about performance	−0.09	−0.91	0.36
UVI x Confidence about performance	0.18	1.81	0.07
Course grade	0.61	5.08	0.00
Enrollment in second course	2.72	12.84	0.00
Baseline major	0.85	4.65	0.00

Note: Coefficients represent log-odds for effects on enrollment and major. All other coefficients are standardized.

Indirect effects.

Given the strong effects of course grade on persistence, it follows that the UVI effect on course grade for FG-URM students might have had long-term implications for persistence. If so, we would expect a significant indirect effect of the UVI on persistence through engagement and performance for these students in this model. Indeed, we found a significant indirect effect of the UVI on enrollment in the second course via engagement and course grade (z = 2.25, p = .02) for all students on average, as well as a significant indirect effect of the three-way UVI x FG x URM interaction on enrollment via engagement and course grade (z = 2.40, p = .02). Conditional indirect effects revealed a significant effect for FG-URM and CG-majority students, but not FG-majority and CG-URM students (see Table 7). Similarly, we found a significant indirect effect of the UVI (z = 2.22, p = .03), as well as the UVI x FG x URM interaction (z = 2.36, p = .02) on biomedical major via engagement, course grade, and enrollment in the second course. We found significant conditional indirect effects for FG-URM and CG-majority students, but not for FG-majority and CG-URM students (see Table 7). These effects suggest that to the extent that the UVI improved engagement and course grades for FG-URM and CG-majority students, it was associated with long-term persistence.

Table 7.

Indirect Effects from the Model Including Personal Focus, Engagement, and Course Grade as Potential Mechanisms of UVI Effects on Persistence in the Biomedical Track

	z	p
UVI → Personal focus x confidence →Enrollment in second course	1.98	0.048
UVI → Personal focus x confidence → Enrollment in second course → Biomedical major	1.95	0.051
UVI → Engagement → Course grade	2.39	0.02
UVI → Engagement → Course grade (CG-Majority)	2.37	0.02
UVI → Engagement → Course grade (CG-URM)	0.03	0.98
UVI → Engagement → Course grade (FG-Majority)	−0.47	0.64
UVI → Engagement → Course grade (FG-URM)	2.43	0.02
UVI x FG x URM → Engagement → Course grade	2.57	0.01
UVI → Engagement → Course grade → Enrollment in second course	2.25	0.02
UVI → Engagement → Course grade → Enrollment in second course (CG-Majority)	2.24	0.03
UVI → Engagement → Course grade → Enrollment in second course (CG-URM)	0.03	0.98
UVI → Engagement → Course grade → Enrollment in second course (FG-Majority)	−0.47	0.64
UVI → Engagement → Course grade → Enrollment in second course (FG-URM)	2.28	0.02
UVI x FG x URM → Engagement → Course grade → Enrollment in second course	2.40	0.02
UVI → Engagement → Course grade → Enrollment in second course → Biomedical major	2.22	0.03
UVI → Engagement → Course grade → Enrollment in second course → Biomedical major (CG-Majority)	2.20	0.03
UVI → Engagement → Course grade → Enrollment in second course → Biomedical major (CG-URM)	0.03	0.98
UVI → Engagement → Course grade → Enrollment in second course → Biomedical major (FG-Majority)	−0.47	0.64
UVI → Engagement → Course grade → Enrollment in second course → Biomedical major (FG-URM)	2.25	0.03
UVI x FG x URM → Engagement → Course grade → Enrollment in second course → Biomedical major	2.36	0.02

To illustrate the implications of these indirect effects, Figure 5 shows the UVI effects as a function of specific course grade categories (Panels A and B) for the full sample and the FG-URM students, respectively, and Panels C and D show the rates of enrollment and biomedical majoring associated with each grade category, for the full sample and specifically for FG-URM students, respectively. Students in UVI conditions (and FG-URM students in particular) were more likely to obtain grades in the AB - B range and less likely to obtain grades in the BC – C range, suggesting that the UVI effect was most pronounced in this range. Higher grades were in turn associated with a higher probability of persisting in the biomedical track, and these differences were most pronounced between the BC – C and AB – B range (i.e., the grade category ranges in which the UVI had the most pronounced effects).

Figure 5.

Descriptive statistics illustrating implications of indirect UVI effects on persistence via course grade. Course grades were higher in the UVI condition than the control condition (Panel A), particularly for FG-URM students (Panel B). Higher grades were in turn associated with higher levels of enrollment in the second course and majoring in a biomedical field, for the full sample (Panel C) and for FG-URM students specifically (Panel D).

Sensitivity analysis for indirect effects via course grade.

Given that we did not find direct effects of the UVI on persistence in the biomedical track that mirrored the UVI effects on course grade (i.e., a main effect of UVI or a UVI x FG x URM interaction), we cannot conclude that grade was a mediator of UVI effects. The indirect effects of the UVI on persistence via course grade point to a potential pathway through which improved grades might promote persistence, but it is possible that an unmeasured confounding variable accounts for these indirect intervention effects. We therefore conducted a sensitivity analysis to test the proportion of variance in the mediator and outcome that would need to be accounted for by an unmeasured confounding variable in order to reduce the indirect effect to zero (Imai, Keele & Yamamoto, 2010).⁹ We found that an unmeasured confounding variable would have to account for at least 13.67% of the variance in both course grade and enrollment in the second course or 19.87% of the variance in both course grade and major in order to reduce the indirect effects of the UVI via course grade to zero. These results suggest that the indirect effects reported on persistence in the biomedical track via course grade are fairly robust to unmeasured confounding variables.

Discussion

We examined the role of a utility value intervention in promoting persistence in the STEM pipeline at the postsecondary level, addressing the need to reduce attrition from biomedical fields. Harackiewicz et al. (2016) tested this intervention in a gateway biology course taken by students intending to major in the biomedical sciences, and we followed these students for two years after they completed the course. Harackiewicz et al. (2016) had found that the UVI improved course grades, averaged across all students and particularly for FG-URM students, and that this intervention effect was mediated by engagement. The present study extended these findings by examining whether the UVI also played a role in promoting students’ persistence in the biomedical track two years beyond this initial course, as well as the psychological processes that may underlie these effects.

We found both direct and indirect effects of the UVI on long-term persistence. As predicted, the UVI had indirect effects by improving course performance, which in turn had strong effects on persistence, and these indirect effects were strongest for FG-URM students. As shown in Figure 5, the intervention was most effective in changing grades between the BC-C and AB - B ranges, and such grade differences can be psychologically significant. A student who receives a C in a critical class may experience their grade as a signal that they can’t make it, whereas a student who gets a B may view their grade as a signal that they are on track. By improving performance in a critical gateway course, the UVI may have helped students meet requirements for their major or increased their belief that they could succeed in a biomedical field. This type of process may be especially important for groups who struggle in STEM courses (e.g., FG-URM students). The UVI partially closed the achievement gap in the introductory biology course for FG-URM students, and the significant conditional indirect effects on course-taking and majoring decisions for this group suggest that UVIs may be a useful tool for promoting diversity in the biomedical fields.

A single intervention in one class may not be sufficient to promote long-term persistence directly for struggling students, but it may get them “over the hump” of gateway courses through this process. However, if subsequent courses are not perceived as valuable, or if other supports are not available (e.g., mentoring, research experiences; Hernandez, Schultz, Estrada, Woodcock, & Chance, 2013; Russell, Hancock, & McCullough, 2007; Schultz et al., 2011), the positive effects of the UVI on confidence may be outweighed by other challenges and dissipate over time. With respect to promoting diversity in the biomedical sciences, the UVI should be viewed as a first step in this multi-stage process. It will be important to consider how the UVI in gateway courses might be combined with “booster” interventions in more advanced courses, or with other mentored research and advising programs for advanced students.

We also found direct effects on persistence, as a function of confidence. Specifically, we found direct effects of the UVI, moderated by students’ confidence levels, on enrollment in a second introductory biology course as well as persistence in a biomedical major two years post-intervention. Simple effects analyses revealed that confidence positively predicted persistence for students who received the UVI, over and above the effect of prior GPA, but not for students in the control condition. These findings suggest that in a context that encourages reflection on the value of course content (created by three writing assignments), students’ confidence in their ability to succeed can affect decisions about whether to persist, but that confidence is less important when the value of the domain is not emphasized.

Moderated mediation analyses indicated that this intervention effect was sequential: within the UVI condition, more confident students were more likely to take the next biology course in the biology sequence, and enrollment in this course in turn led to long-term pursuit of a biomedical major. This finding is consistent with theorizing on how more proximal intervention effects may be “channeled” by the social system to affect longer term outcomes (Cohen, Garcia, & Goyer, 2017; Goyer et al., 2017). The UVI created a context that emphasized value, which led more confident students to complete another biology course that could help to fulfill biomedical major requirements, facilitating progress on an educational trajectory that increased the likelihood of persistence in a biomedical major.

The Power of Linguistic Analysis

Linguistic analysis permitted us to measure both personal focus and engagement. Importantly, we found that the direct intervention effects on long-term persistence were partially mediated by personal focus, an index of the degree to which students reflected, in their essays, on the personal relevance of biological topics in the introductory course. The UVI promoted reflection on the personal relevance of course material for all students, on average, and this increased personal focus in turn amplified the role of confidence in determining pursuit of a biomedical education. By helping students to appreciate the value of knowledge in the biomedical domain, the intervention seemed to catalyze the influence of students’ beliefs about whether they could succeed in the domain when deciding whether to persist. This finding is consistent with the positive expectancy-value interaction posited by expectancy-value theory (Eccles & Wigfield, 2002; Nagengast et al., 2011), which presumes that students who both expect to succeed in and personally value a domain are the most likely to pursue an education in that domain. However, this process may be a double-edged sword. In an introductory college course, students are making decisions about what field they want to pursue. An intervention that helps students focus on the personal value of biology may make confident students more likely to pursue biomedical majors, but for students who already have some doubts, focusing on the personal value of biology may lead them to decide they would rather pursue another field (e.g., a field in which they perceive both value and a high likelihood of success).

Critically, we found that the process through which the UVI influenced long-term persistence was distinct from the process by which it promoted course grades for disadvantaged students. Whereas the direct effect on persistence was mediated by personal focus (and not by engagement), the direct effect on course grades for FG-URM students (and for all students, on average) was mediated by engagement with course content (and not by personal focus). These findings suggest that the UVI may initiate two distinct processes: (a) helping underperforming students to engage with course material, thereby improving performance and helping them believe they can pursue the domain, and (b) increasing reflection on the personal relevance of course material, thereby helping more confident students to see why pursuing that domain may be worthwhile.

Implications for Intervention Science

These findings underscore a critical principle of intervention science: interventions target specific outcomes through specific motivational processes (Harackiewicz & Priniski, 2018). Although the original study focused on improving performance, the current study focused on persistence, and thus different processes were critical to this outcome. In the case of the UVI, we found that a single intervention influenced two separate processes (engagement with course material and reflection on personal relevance) and these processes led to effects on distinct educational outcomes for different groups of students. Our findings suggest that researchers may benefit from identifying the most relevant outcome variables and considering whether an intervention may initiate multiple processes, which may in turn influence different academic outcomes. Furthermore, the current study builds upon prior work showing that linguistic analysis can provide insight into the cognitive processes at play as students complete an intervention (e.g., Beigman Klebanov et al., 2017; Harackiewicz et al., 2016; Priniski et al., in press). Many interventions involve a writing component, and the words students use represent a rich and underutilized data source in intervention science (Harackiewicz & Priniski, 2018).

Our results suggest that the UVI initiates multiple processes that culminate in increased performance in the course and increased persistence in the biomedical track in the long term. However, it is important to note that although the UVI has overall positive effects, some groups benefit more than others. First and foremost, the UVI improves course grades, for all students on average, for students with a history of poor performance, and for FG-URM students. The indirect effects observed in the current study suggest that these positive effects on grades may translate into long-term increases in persistence in the biomedical track. These results suggest that encouraging students to reflect on the utility value of topics in introductory STEM courses may be an effective way to promote persistence in the STEM pipeline for students who enter the course with a lower likelihood of succeeding, by helping them to obtain higher grades.

The current study also indicates that there is a parallel process unfolding over time as a function of students’ confidence. The UVI creates a context in which students focus on the personal value of biology. In this context, students who are more confident in their ability to succeed in biology are more likely to persist in the biomedical track, which is consistent with expectancy-value theory. It is important to note that the simple effects of the intervention were not significant at high or low levels of confidence, suggesting that it is not the case that the UVI was good for some students and bad for others. Rather, the UVI amplified the effects of confidence, such that for students who received the UVI, confident students were more likely to persist in the biomedical track. One open question that will need to be answered in future studies is how this process would play out in introductory courses for non-majors in which initial levels of confidence and value span a broader range. Students in the current study were all at least considering a major in the biomedical sciences. It is unclear whether the UVI would interact with confidence in the same way in a general education course in which most students are not intending to major in that field.

In sum, the UVI is primarily a curricular intervention intended to promote engagement and performance in a particular course. A growing body of research suggests that this intervention promotes engagement and performance in introductory courses and has especially strong effects for the students who need it most (Harackiewicz & Priniski, 2018; Walton & Wilson, 2018). In addition, the results of the current study suggest that the UVI is an intervention with the potential to increase persistence in the biomedical sciences, in two ways. First, the UVI increases grades in important gateway courses for all students on average and FG-URM student in particular, and these higher grades result in a greater chance of persistence. Second, the UVI creates a context in which students focus on the personal value of biology, which (at least in an introductory course for biomedical majors) makes more confident students more likely to persist. Thus, the results of the current study help to clarify, on theoretical and practical levels, which groups are likely to benefit from the UVI in terms of performance and persistence. This provides important insights for educators and researchers who may be interested in implementing a UVI, and provides a model for intervention scientists to investigate the multiple processes and outcomes that may be impacted by a single intervention.

Limitations

Although this follow-up study allowed us to examine the influence of the UVI on college biology students’ long-term educational trajectories, there are several notable limitations. First, although the UVI promoted long-term persistence, our findings are limited to the context of the biomedical fields. It will be critical for future studies to investigate whether these effects generalize to other STEM fields. Second, although we provide evidence of intervention effects on students’ persistence in a biomedical major two years post-intervention, a next step for future studies would be to examine whether this influence can translate into even longer-term effects in the STEM pipeline (e.g., career choices). Finally, because we found indirect, but not direct, effects of the intervention on long-term persistence in the biomedical track for FG-URM students, we cannot draw causal inferences about the potential of the UVI to increase diversity in the STEM fields. There are several possibilities that may explain our pattern of results. One possibility is that long-term direct effects for FG-URM students could not be detected because the positive effect through course grade was suppressed by negative effects through unmeasured mechanisms (Rucker, Preacher, Tormala, & Petty, 2011). Another possibility is that because these indirect effects are correlational, other external variables could underlie these long-lasting effects rather than the intervention. The results of our sensitivity analysis suggest that the indirect effects reported here were fairly robust to unmeasured confounding variables, but this possibility is nevertheless important to consider. Indirect effects are important to examine because they allow for investigation of the specific processes through which the proximal effects of an intervention may unfold over time to produce long-term impacts (Cohen, Garcia, Purdie-Vaughns, Apfel, & Brzustoski, 2009; Harackiewicz & Priniski, 2018; Hecht, Priniski, & Harackiewicz, in press; MacKinnon, Krull, & Lockwood, 2000; Rozek, Svoboda, Harackiewicz, Hulleman, & Hyde, 2017; Tibbetts et al., 2016). However, we interpret these indirect effects not as concrete evidence, but rather as a pattern that is consistent with our reasoning that the UVI may have lasting impacts on persistence in the biomedical track for underrepresented students by improving their performance in introductory courses.

Conclusion

A growing body of research indicates that well-timed, theoretically-guided social-psychological interventions can affect important educational outcomes (Harackiewicz & Priniski, 2018; Walton & Wilson, 2018). The current study contributes to this research by showing that implementing a UVI in introductory STEM courses can affect long-term persistence in these fields. Furthermore, our results highlight the importance of thinking about interventions not as silver bullets that work through mysterious black boxes, but as precise strategies for addressing specific problems by targeting the relevant psychological processes (Harackiewicz & Priniski, 2018; Walton & Wilson, 2018). In sum, this study highlights the potential of a utility-value intervention to promote persistence in the STEM pipeline, and lends insight into the processes through which such long-term effects unfold over time.