Choose Your Own Intervention: Using Choice to Enhance the Effectiveness of a Utility-Value Intervention

Abstract

Utility-value interventions, in which students are asked to make connections between course material and their lives, are useful for improving students’ academic outcomes in science courses. These interventions are thought to be successful in part because the intervention activities afford students autonomy while they complete them, but no research has explored directly whether interventions that include more support for autonomy are more effective. In this study, the degree of choice incorporated in a utility-value intervention was systematically varied in order to test this possibility. We assigned college biology students (n = 406) to a high-choice utility-value intervention condition (choose between two formats- essay or letter- for each of 3 writing assignments), one of two low-choice intervention conditions (complete either an essay and then a letter, or vice versa, and choose a format for the third assignment), or a control condition (summarize course material 3 times). Students in the high-choice condition reported significantly higher perceived utility value and interest for biology course content compared to students in the low-choice conditions. There were also significant, but small, indirect effects of choice on students’ final course grades and enrollment in the next course in the biology sequence via perceived utility value and interest. Results suggest that social-psychological interventions which include more choice are likely to be more effective than those which include less choice.

There is a growing need for students to have knowledge and skills in the subject areas of science, technology, engineering, and mathematics (STEM) across a variety of occupations (National Science Board, 2014). However, only about half of college students who intend to pursue a bachelor’s degree in a STEM field do so, and therefore many students do not acquire these important skills (Chen, 2013). One way to enhance students’ interest and performance in STEM fields is by incorporating motivation interventions in college STEM courses (Harackiewicz & Priniski, 2018; Rosenzweig & Wigfield, 2016). In particular, researchers working within Eccles and colleagues’ expectancy-value theory of motivation (Eccles et al., 1983) have conducted utility-value (UV) interventions, to increase students’ perceived value of an academic domain by asking them to write about how course material relates to their lives (Hulleman & Harackiewicz, 2009). Expectancy-value theory posits that students’ perceptions of the utility value of an academic task are important predictors of their academic motivation, engagement, and achievement (Harackiewicz, Tibbetts, Canning & Hyde, 2014; Wigfield, Rosenzweig, & Eccles, 2017). To date, UV interventions have promoted students’ interest, course achievement, and subsequent course enrollment (e.g., Canning et al., 2018; Harackiewicz, Canning, Tibbetts, Priniski, & Hyde, 2016; Hulleman, Godes, Hendricks, & Harackiewicz, 2010; Hulleman, Kosovich, Barron, & Daniel, 2017).

In order to make UV interventions as effective as possible, it is critical to examine how different components of these interventions work. One aspect that warrants study is the degree of choice afforded by the intervention. Yeager and Walton (2011) have argued that students respond more positively to social-psychological interventions when they are able to internalize an intervention’s message, instead of feeling pushed to endorse a particular attitude or belief. This reasoning is grounded in self-determination theory (Ryan & Deci, 2000), which posits that students who perceive more autonomy will engage more deeply with their learning and enjoy it more. In educational settings, giving students choices about learning tasks is a common way to promote perceptions of autonomy. Research shows that when students are offered meaningful choices, they are more interested and engaged while completing academic tasks (see Patall, Cooper, & Robinson, 2008, for a review).

UV interventions almost always include some choices: students can choose what scientific topic to write about, what connections to make to course material, whether to write about utility value for themselves or for others, and in some cases, what type of writing task to complete (e.g., choose to write in an essay format or a letter format; Canning et al., 2018; Harackiewicz et al., 2016). Having these choices likely makes UV interventions more effective than if the choices were not included, because students are likely to become more engaged if they think about how course material relates to their lives on their own terms. Higher engagement and perceived autonomy during the intervention may help students benefit more from the intervention, thus perceiving course material as more valuable and interesting. However, no research has systematically varied the amount of choice students receive during UV interventions to explore whether this is the case.

In the present study, we examined whether college biology students who received a high-choice UV intervention had higher perceived utility value, interest, course grades, or enrollment rates in the next course in the biology sequence, compared to students who received either of two low-choice interventions, or to a control condition. We hypothesized that students in the high-choice condition, versus the low-choice conditions, would report higher interest and value for learning their course material. In turn, we expected that perceiving higher interest and value would lead to higher course grades and subsequent course enrollment.

Method

Participants and Design

Four hundred six students who were enrolled in an introductory biology course at a large U.S. university (58.6% female; 84% White; 16% Asian or Asian-American; mean age = 19.19 (0.76)) comprised the sample for the present study. There were 843 students enrolled in the course, all of whom participated in a broader research project that examined UV interventions in biology (see Harackiewicz et al., 2016; Canning et al., 2018 for more detail about the project; the project was IRB approved). Some students, including all first-generation college students and underrepresented racial/ethnic minority students, were not eligible for this study because they had been assigned to experimental conditions testing a different research question during the same semester (see online supplement for more information). There were 424 students who were eligible for this study; 406 gave consent and completed the course. Students were randomly assigned to one of four conditions: a high-choice UV intervention condition, one of two low-choice UV intervention conditions, or a control condition.

Materials and Procedure

Students completed questionnaires during the second week of the semester and again during the last week of the semester. Interventions were delivered via three writing assignments, evenly spaced over the semester. Students were assigned randomly to condition prior to the first writing assignment, and both students and professors were blind to students’ condition.

Intervention and control materials.

For each of the three writing assignments, all students were asked to write a 1–2 page paper about a topic they had been studying in biology by formulating a scientific question and answering it. In the high-choice condition, students could choose on all three assignments whether to write in one of two formats - an essay or a letter. In the essay format, students wrote about how course material related to their own lives, and in the letter format, students wrote a letter to a friend or family member about how course material related to that person’s life. In the two low-choice intervention conditions, students either were assigned to complete assignment 1 in the letter format and assignment 2 in the essay format (the letter-essay low-choice condition), or vice versa (the essay-letter low-choice condition). Across all three UV intervention conditions students decided which topics to write about, and they all were given a choice between the essay and the letter format on assignment 3. Students who had choices chose the letter and the essay with similar frequencies: In the high-choice condition, 51% of students wrote mostly essays (either two or three essays across the three assignments) and 49% wrote mostly letters (either two or three letters); in the low choice conditions, on the third assignment, 55% of students wrote a letter, and 45% wrote an essay. In the control condition, students wrote essays summarizing their chosen topic on all three assignments.

Measures.

We assessed students’ confidence to learn biology (3 items; α = .83) at the beginning of the semester. At the end of the semester, we measured students’ perceived utility value (12 items; α = .88) and interest (9 items; α = .83), for the topics covered in the biology course. All items were answered on a 1–7 Likert-type scale. Academic outcomes included students’ course grades and enrollment in the next course in the biology sequence. Prior achievement was measured in terms of students’ cumulative grade point average (GPA) at the end of the previous semester; all academic data were collected via institutional records.

The online supplement contains the self-report items and the intervention and control prompts.

Results

Descriptive Statistics and Correlations

Descriptive statistics and correlations among major variables are reported in Table 1.

Table 1.

Descriptive Statistics and Correlations

Variable	N	M	SD	SE	1.	2.	3.	4.	5.
1. Baseline perceived competence	401	5.84	0.89	0.04
2. Prior GPA	392	3.24	0.61	0.03	.08
3. Overall utility value	396	5.33	0.95	0.05	.21**	.15**
4. Overall interest	399	5.16	0.99	0.05	.28**	.15**	.67**
5. Final grades	401	84.07	6.70	0.33	.16**	.58**	.31**	.34**
6. Enrollment in next biology course	406	81%	—		.12**	.17**	.27**	.22**	.25*

Note. GPA = grade point average.

^*p < .05;

^**p < .01

Primary Analyses: Effects of High-Choice versus Low-Choice Intervention Conditions

Analysis strategy.

We used multiple linear regression (logistic regression for the course enrollment measure) with three orthogonal contrasts: (i) Overall UV (each UV intervention condition coded as +1; control condition coded as −3); (ii) High Choice (high-choice UV intervention condition coded as +2; each low-choice UV intervention condition coded as −1; control condition coded as 0); (iii) Assigned Letter Format First (letter-essay low-choice UV intervention condition coded as +1; essay-letter low-choice UV intervention condition coded as −1; high-choice and control conditions coded as 0). The final model included 8 terms: three contrast codes, covariates of baseline perceived competence in biology (standardized), gender (male = −1; female = +1) and prior GPA (standardized), and two dummy variables to account for students’ nesting within one of three lecture sections. All analyses were conducted in MPLUS version 7 using full information maximum likelihood estimation for missing data.¹

Primary analyses.

Mean scores by condition are reported in Table 2, and results of regression analyses are reported in Table 3. There were significant effects of the High Choice contrast on perceived utility value for biology, β = 0.17, p < .001, and interest in biology, β = 0.11, p = .02. We did not find significant main effects of the Overall UV or Assigned Letter Format First contrasts on any outcomes.

Table 2.

Descriptive Statistics by Study Condition

Measure	High Choice			Essay-Letter Low Choice			Letter-Essay Low Choice			Control
	M	SD	SE	M	SD	SE	M	SD	SE	M	SD	SE
Baseline perceived competence	5.78	0.85	0.09	5.88	0.91	0.09	5.89	0.90	0.09	5.83	0.91	0.08
Prior GPA	3.29	0.62	0.07	3.21	0.66	0.07	3.24	0.62	0.06	3.24	0.57	0.05
Overall utility value	5.64	0.82	0.09	5.24	0.97	0.10	5.21	1.01	0.10	5.26	0.93	0.09
Overall interest	5.38	0.87	0.09	5.11	0.98	0.10	5.12	0.97	0.10	5.07	1.08	0.10
Final grades	84.39	7.25	0.76	83.23	6.85	0.71	84.31	7.49	0.76	84.29	5.30	0.49
Enrollment in next biology course	81%			80%			80%			83%

Note: GPA = grade point average.

Unadjusted mean scores shown

Table 3.

Regression Results from Overall Model

Predictor Variable	Perceived Utility Value			Interest			Final Grades			Enrollment in Next Biology Course
	β	S.E.	z	β	S.E.	z	β	S.E.	z	b	S.E.	z
Overall UV contrast	0.05	0.05	0.97	0.06	0.05	1.19	−0.03	0.04	−0.66	−0.05	0.08	−0.66
High Choice contrast	0.17**	0.05	3.60	0.11*	0.05	2.27	0.02	0.04	0.61	0.01	0.11	0.06
Assigned Letter Format First contrast	0.004	0.05	.083	−0.004	0.05	−0.10	−0.05	0.04	−1.18	0.00	0.18	−0.002
Baseline perceived competence	0.23**	0.05	4.60	0.29**	0.05	6.07	0.11*	0.04	2.55	0.38**	0.12	3.10
Gender	0.08	0.05	1.58	0.06	0.05	1.29	−0.03	0.04	−0.74	0.34*	0.14	2.51
Prior GPA	0.14**	0.05	2.73	0.14**	0.05	2.83	0.56*	0.04	16.25	0.38**	0.13	2.91

Notes. GPA = grade point average.

^*p < .05;

^**p < .01.

Overall UV Contrast = UV Intervention versus control condition (each UV intervention condition = +1; control condition = −3); High Choice Contrast = high-choice versus low-Choice UV intervention conditions (high-choice condition = +2; each low-choice condition = −1); Assigned Letter Format First Contrast= letter-essay versus essay-letter low-choice UV intervention conditions (letter-essay condition = +1; essay-letter condition = −1). Gender = female (+1) versus male (−1). n = 406 for all analyses. Enrollment in next biology course was estimated using logistic regression. Students’ course section was controlled for in all analyses but is not reported in the table

There were no main effects of High Choice on grades or course enrollment (see Table 3). However, we had hypothesized that choice would affect students’ course outcomes as a result of increasing perceived utility value and interest. We therefore tested for indirect effects of the High Choice contrast on students’ final grades and course enrollment through students’ perceptions of utility value or interest, using path analysis in MPlus. Indeed, there were significant indirect effects of High Choice on course grades via both perceived utility value and interest. On subsequent course enrollment, there was a significant indirect effect of High Choice via perceived utility value, but the effect via interest was not significant (see Table 4). That is, receiving the high-choice UV intervention (versus low choice) predicted students having higher interest and utility value, and these students’ interest and utility value were associated with them earning higher grades and (for utility value) having a higher probability of course enrollment.

Table 4.

Indirect Effects of High Choice Contrast on Course Outcomes

	Final Grades			Enrollment in Next Biology Course
Process Variable	Estimate	SE	z	Estimate	SE	z
Perceived utility value	0.04**	0.01	3.08	0.09**	0.03	2.87
Interest	0.03*	0.01	2.15	0.04†	0.02	1.89

Notes.

^*p < .05;

^**p < .01;

^†p < .10.

Estimates are standardized for final grades and unstandardized for enrollment in next biology course.

Consistency of Utility Value Intervention Effects with Previous Research

Previous research testing UV interventions in college courses has found evidence for main effects on academic outcomes (perceived value, interest, performance, continuation) or negative interactions with prior performance and/or perceived competence indicating that UV interventions are particularly helpful for students who struggle or doubt their competence (Harackiewicz & Priniski, 2018; Tibbetts, Harackiewicz, Priniski, & Canning, 2016). We did not find evidence for either pattern with the Overall UV contrast in the current study. Research has also shown UV interventions to be particularly helpful for underrepresented students (i.e., students who are first-generation and members of underrepresented minority groups; Harackiewicz et al., 2016), but there were no underrepresented students in the current study. One reason why we did not find stronger effects of the UV intervention in this study may be because our sample did not include the underrepresented students who are likely to benefit most strongly from UV interventions in biology.

Another possible explanation is that prior UV intervention studies in science have offered a majority of students high amounts of choice, whereas our test of an overall UV effect was based on two-thirds of the students receiving low-choice conditions. In the two prior UV intervention studies that have been done in college biology, Canning and colleagues (2018) and Harackiewicz and colleagues (2016) gave at least half of their intervention group students two or three choices between writing letters and essays, using the same choices that were offered in the present study (Canning et al., 2018; Harackiewicz et al., 2016). Additionally, Hulleman and Harackiewicz (2009) gave high school science students an average of four to five assignments, in which intervention group students were given choices between writing about utility value for the self or utility value for a family member or friend. In order to examine how our results compared to what has been found in these prior studies, we conducted further analyses comparing only the high-choice intervention condition to the control condition. For this analysis we utilized the same regression model as above, but only examined data from the high-choice and control conditions (n = 213; high-choice condition coded as +1; control condition coded as 0). We also retained interactions between the high-choice versus control term and students’ baseline perceived competence and GPA in these models, for consistency with prior studies.

Results (see Table 5) showed that students in the high-choice condition reported higher perceived utility value, β = 0.20, p = .002, and interest, β = 0.14, p = .03, compared to students in the control condition. There were also indirect effects (see Table 6) suggesting that the high-choice condition (versus control) promoted students’ utility value, and these students’ higher utility value predicted them earning higher final grades, p = .03 (the indirect effect on course enrollment via utility value was at significance, p = .054). These findings are generally consistent with previous research (e.g., Canning et al., 2018; Hulleman et al., 2010). However, we did not find significant direct effects of the high-choice versus control comparison on students’ grades or course enrollment or any significant interactions of the high-choice versus control comparison with students’ baseline perceived competence or GPA. In sum, the high-choice condition tested in this study showed fewer positive effects than prior studies; however, the significant main effects reported here were consistent with prior work.

Table 5.

High-Choice versus Control Regression Results

Predictor Variable	Perceived Utility Value			Interest			Final Grades			Enrollment in Next Biology Course
	β	S.E.	z	β	S.E.	z	β	S.E.	z	β	S.E.	z
High-choice vs. control	0.20**	0.07	3.10	0.14*	0.06	2.24	−0.01	0.06	−0.21	−0.21	0.37	−0.56
Baseline perceived competence (BPC)	0.25**	0.09	2.84	0.32**	0.08	3.85	−0.03	0.07	−0.42	0.31	0.23	1.39
Gender	0.10	0.07	1.45	0.05	0.07	0.74	−0.11†	0.06	−1.91	0.28	0.19	1.46
Prior GPA	−0.04	0.10	−0.39	0.03**	0.09	0.37	0.48	0.07	6.51	0.28	0.25	1.14
High-Choice x BPC	−0.09	0.09	−0.96	−0.07	0.09	−0.80	0.07	0.08	0.97	−0.19	0.38	−0.51
High-Choice x GPA	0.12	0.09	1.33	0.11	0.09	1.19	0.14†	0.08	1.78	0.003	0.36	0.01

Notes.

^*p < .05;

^**p < .01;

^†p < .10.

High-choice versus control comparison = high-choice condition (+1) versus control(0). Gender = female (+1) versus male (−1). n = 213 for all analyses. Enrollment in next biology course was estimated using logistic regression. Students’ course section was controlled for in all analyses but is not reported in the table

Table 6.

Indirect Effects of High-Choice versus Control on Course Outcomes

	Final Grades			Enrollment in Next Biology Course
Process Variable	Estimate	SE	z	Estimate	SE	z
Perceived utility Value	0.04*	0.02	2.16	0.21*1	0.11	1.93
Interest	0.03†	0.01	1.83	0.12	0.08	0.14

Notes.

^*p < .05;

^**p < .01;

^†p < .10;

^*1p = .054.

Estimates are standardized for final grades and unstandardized for enrollment in next biology course.

Discussion

Students who had more choices during a UV intervention reported higher interest and perceived utility value for biology than those who had fewer choices. This study is the first to test choice as a moderator of a UV intervention, extending prior research about the positive effects of choice in classroom settings (Patall et al., 2008). Results suggest that UV interventions which have more choices are more effective at improving interest and value compared to interventions which include fewer choices. In the present study, the interventions were not effective unless they included high amounts of choice. We do not believe that all UV interventions need high amounts of choice to be effective, because other studies have shown that UV interventions promoted students’ achievement and interest without including as much choice as our high-choice condition (e.g., Hulleman et al., 2010; Hulleman et al., 2017). In other course contexts, perhaps those where students have more autonomy over their learning than the highly-regulated biology course, utility-value interventions may be effective even with less choice. We conclude that when possible, researchers should add more choices into UV interventions because they will be more effective than if the interventions included fewer choices. Effects of choice were small, but the choices offered to students required minimal changes to the intervention. The choices also did not change the fundamentals of the assignments (relate course material to their own life or the life of a friend or family member), and in both the high- and low-choice conditions students were exposed to both assignment formats.

Another strength of this work is that it integrates expectancy-value and self-determination theories in the intervention setting. Our results suggest that when students have more choices, they may be more receptive to interventions that try to boost their task value. Choice also may be useful for improving the effectiveness of other expectancy-value-based interventions in future studies.

There were indirect effects of choice on students’ course grades and enrollment in the next course, through perceived utility value and, to a lesser extent, interest. Results do not show conclusively that students who receive more choices during an intervention have higher grades and course enrollment. However, the results do suggest an interesting potential process through which students who have more choices perceive more value in their coursework, which ultimately might help them to achieve higher grades and continue in a field. This pathway has implications for understanding how to improve students’ course grades and participation in STEM fields moving forward, so it is important for researchers to explore whether having more choices in other contexts boosts students’ academic outcomes directly.

Future work should build on our findings in several ways. First, in this study students chose between a personal essay or a letter to a close friend or family member, and all students chose how to connect material to one’s life. Personal relevance is thought to be a useful way to promote autonomy in addition to choice, so the intervention activities already provided a fairly autonomy-supportive context to students (e.g.., Assor, Kaplan, & Roth, 2012; see Katz & Assor, 2007, for review). Additionally, the choice offered was designed to be optimally effective for students, because it was personally meaningful but not overly taxing for students to think about (see Katz & Assor, 2007; Patall et al., 2008, for reviews). Researchers should explore whether other types of choices are less effective than the choices used in this study, such as choices which require participants to evaluate many options, choices that are highly superficial with no meaning to the participants, or choices that occur in an otherwise highly-controlled intervention environment. Researchers might also explore whether it is more powerful to offer students a choice of whether or not to engage in UV writing at all, rather than to offer a choice between two different UV writing activities, as this might promote students’ perceptions of autonomy even more (Reeve, Nix, & Hamm, 2003; see Katz & Assor, 2007, for review). Second, our sample was not fully representative of college students; these findings are intended to demonstrate basic motivational processes that might apply to all students, but future work should explore directly whether our results generalize to underrepresented students. Third, we recommend that researchers more directly examine the mechanisms by which choice can help students perceive more utility value and interest (e.g., increased perceptions of autonomy, increased engagement). With more work exploring these critical processes, it may be possible to use choice to improve the effectiveness of other motivation interventions in STEM contexts.