Abstract
The current study reports on the efficacy of a multi-faceted motivationally designed undergraduate enrichment summer program for supporting science, technology, engineering and math (STEM) persistence. Structural equation modeling was used to compare summer program participants (n = 186), who participated in the program between their first and second years in college, to a propensity score matched comparison sample (n = 401). Participation in the summer program positively predicted science motivation (self-efficacy, task value), assessed eight months after the end of the program (second year in college). The summer enrichment program was also beneficial for science persistence variables, as evidenced by significant direct and indirect effects of the program on science course completion during students’ third year of college and students’ intentions to pursue a science research career assessed during the third year of college. In general, the program was equally beneficial for all participants, but ancillary analyses indicated added benefits with respect to task value for students with relatively lower prior science achievement during the first year of college and with respect to subsequent science course taking for males. Implications for developing effective interventions to reduce the flow of individuals out of STEM fields and for translating motivational theory into practice are discussed.
Keywords: expectancies, intervention, motivation, science persistence, task value
Many students enter college with an interest in studying science. However, after enrolling in introductory-level science courses, students frequently abandon this goal and choose a different path. Fewer than 40% of students who begin college with an interest in science complete a degree in Science, Technology, Engineering, or Mathematics (STEM) fields (President’s Council of Advisors on Science and Technology [PCAST], 2012). Even more concerning, there is continued underrepresentation of racial/ethnic minorities and women in STEM fields (Chen, 2013; National Science Foundation, 2015; PCAST, 2012). Among other factors, this “leaky pipeline” at the undergraduate level contributes to concerns over shortages in qualified candidates for science-research jobs as well as broader concerns regarding the scientific literacy of our nation. As a result, there is a growing movement within higher education institutions to address the underlying institutional contributors to STEM attrition (Fry, 2014).
Much of the prior research on STEM persistence sought to identify the qualities of individuals who successfully persist in STEM fields. Using both concurrent and retrospective reports, this research finds that individuals who continued in science cited reasons like enjoyment, curiosity, and the desire to help others as key reasons for their persistence in STEM fields (McGee & Keller, 2007; Rayman & Brett, 1995; Seymour, 1995). A complimentary body of research focuses more specifically on how particular types of experiences, such as undergraduate research experiences or university-based enrichment programs, support undergraduate STEM persistence (e.g., Hernandez, Schultz, Estrada, Woodcock, & Chance, 2013; Jones, Barlow, & Villarejo, 2010; Maton, Domingo, Stolle-McAllister, Zimmerman, & Hrabowski, 2009; Russell, Hancock, & McCullough, 2007; Schultz et al., 2011; Stolle-McAllister, Domingo, & Carillo, 2011; Villarejo, Barlow, Kogan, Veazey, & Sweeney, 2008). Much of this early work employed either concurrent or retrospective reports of alumni who participated in enrichment programs or experiences, with only a few large-scale prospective studies providing initial evidence that students who participate in undergraduate enrichment experiences are more likely to persist in STEM (e.g., Hernandez et al., 2013; Jones et al., 2010; Schultz et al., 2011).
What is often missing from these approaches, however, is a consideration of the psychological mechanisms involved in STEM persistence (although see Hernandez et al., 2013 for an exception) and the ways in which enrichment experiences or other institutional supports can be designed with the explicit goal of supporting science motivation and, ultimately, STEM persistence. In the current study, we apply a motivational lens to the study of STEM persistence. Specifically, we approach the issue of STEM persistence through supports for undergraduates’ motivational beliefs in science using an ecologically valid summer enrichment program explicitly designed to support multiple types of motivational beliefs. Thus, our perspective provides an alternative, motivational approach to STEM persistence that considers motivational beliefs as a key intermediary process and serves as the foundation for developing effective enrichment experiences. A motivational approach for addressing the “leaky pipeline” is particularly useful as students who have the skill and initial desire to pursue STEM careers often leave because they no longer believe they have the skills to be successful or no longer find the field interesting or personally valuable (e.g., Seymour & Hewitt, 1997). Furthermore, this type of systematic investigation of how to support science motivation under ecologically valid conditions provides valuable insights regarding the translation of motivational theory into educational practice. The current study also takes an alternative approach to prior, more targeted motivational approaches (e.g., utility value interventions, Harackiewicz, Canning, Tibbetts, Priniski, & Hyde, 2016) to consider whether a multi-faceted, holistic approach to designing an undergraduate educational experience can help to support multiple forms of motivation and subsequent persistence in STEM fields. Such an approach addresses a call from researchers to translate motivational research into practice by supporting multiple forms of motivation in authentic educational contexts (Linnenbrink-Garcia, Patall, & Pekrun, 2016; Pintrich, 2003).
Theoretical Framework
When taking a motivational approach to supporting STEM persistence, it is critical to ground one’s approach within a theoretical framework in order to clearly conceptualize how the intervention is expected to operate and to evaluate its effectiveness (Rosenzweig & Wigfield, 2016). In the current study, we utilized Eccles’s Expectancy-Value Theory of motivation (Eccles et al., 1983; Wigfield & Eccles, 2002). This theory incorporates students’ beliefs about their own abilities and their perceptions of value for a particular domain or task. According to modern expectancy-value theory, expectancies for success (e.g., perceived competence) and task value are critical predictors of academic achievement and academic/career choices (Eccles et al., 1983) and thus are important predictors of key outcomes related to STEM persistence.
Expectancies refer to students’ beliefs about whether they will be capable of succeeding at a task; they are part of a larger family of competence-related beliefs such as self-efficacy, self-concept, and perceived competence. Prior research suggests that constructs from this broad family of competence beliefs are the strongest predictors, relative to other types of motivational beliefs, of academic achievement across a variety of disciplines (Schunk & Pajares, 2005; Wigfield & Cambria, 2010). Competence beliefs are also associated with choice behaviors such as decisions to persist, although to a lesser extent than task value (Wigfield & Cambria, 2010). The second component of expectancy-value theory is task value, which is typically divided into four types of value (Eccles et al., 1983): (1) intrinsic value, finding a particular task or domain enjoyable and interesting; (2) utility value, the usefulness of a task or subject area; (3) attainment value, the personal importance of doing well on a particular task or in a particular domain for one’s identity; and (4) cost, the perceived drawbacks of engaging in a particular task or domain due to high effort needed for success, lost opportunities to engage in other tasks or domains, and psychological or emotional costs. Value-related beliefs are predictive of achievement and academic engagement (Schiefele, 2001), but are even stronger predictors of choice behaviors such as career aspirations (Wigfield & Cambria, 2010; Wigfield & Eccles, 2002).
Drawing from expectancy-value theory, we would expect that high perceived competence and task value in STEM domains are each important, proximal factors that help to encourage students to pursue careers in STEM fields. Heightened perceived competence should help students better cope with challenging undergraduate STEM courses, leading to high levels of achievement in STEM courses. Task values are expected to be essential determinants of students’ decisions to pursue science course work and their career-related beliefs in science. Empirical support for the expectancy-value framework in relation to STEM retention has accumulated in recent years. In particular, students who value science more (and perceive lower costs) are more likely to stay in STEM fields in college (Andersen & Ward, 2014; Estrada, Woodcock, Hernandez, & Schultz, 2011; Maltese & Tai, 2011; Perez, Cromley, & Kaplan, 2014; Wang & Degol, 2013). Low or declining self-efficacy in STEM early in college is also associated with STEM attrition (Larson et al., 2015; Raelin et al., 2014), though this may not be the case after controlling for value (Andersen & Ward, 2014). Taken together, these findings provide support for the idea that enhancing perceived competence and task value may be particularly useful for supporting undergraduates’ persistence in STEM fields.
Supporting expectancies and values
Given the importance of expectancies and task value in supporting STEM persistence, one potential avenue for intervening to support STEM persistence is to target students’ motivational beliefs directly. Though not as extensive as research drawing from other motivational theories to support motivation, a number of studies have sought to shape competence beliefs and task value in educational settings (for reviews see Harackiewicz & Priniski, 2018; Hulleman & Barron, 2016; Lazowski & Hulleman, 2016; Rosenzweig & Wigfield, 2016). One prominent approach, a utility-value intervention, was developed by Harackiewicz, Hulleman, and their colleagues. In this approach, instructors are asked to implement one or more carefully-designed writing assignments in which students are asked to write about the usefulness or relevance of their coursework. Utility value interventions have been implemented at both secondary and post-secondary levels and are associated with increased STEM-related interest, course-taking, and achievement, and appear to be especially beneficial for students with low perceived competence or prior achievement (e.g., Hulleman & Harackiewicz, 2009) and those who are traditionally underrepresented in STEM fields such as first-generation and underrepresented minority students (Harackiewicz et al., 2016).
Other motivational interventions seek to alter student motivation by changing the classroom or school context. Some classroom interventions target a specific form of motivation such as self-efficacy. For example, Siegle and McCoach (2007) found that students of fifth grade mathematics teachers trained to promote self-efficacy through instructional strategies (e.g., goal setting, teacher feedback, and modeling) had significantly higher self-efficacy in mathematics at the end of the 4-week unit, which was in turn associated with higher mathematics achievement. Relatively fewer interventions focus on altering multiple forms of motivation. For instance, Feng and Tuan (2005) developed a unit on acids and bases for 11th grade Taiwanese students with low motivation in chemistry using Keller’s ARCS (Attention, Relevance, Confidence, and Satisfaction) model, which is based on expectancy-value theory to support perceived competence and values. Compared to a control group, students in the ARCS condition had higher self-efficacy in science, higher value for science, greater use of active learning strategies, and higher achievement during the unit.
One of the potential benefits of motivational interventions that target more than one psychological variable (e.g., both competence beliefs and values) is that prior research suggests that these two forms of motivation are both useful but differentially predict key outcomes associated with STEM persistence (e.g., achievement and choice behaviors, Wigfield & Cambria, 2010). Yet, relatively few studies examine the effectiveness of educational contexts explicitly designed to holistically support multiple forms of student motivation. Thus, for this study, we developed a motivational intervention designed to enhance both self-efficacy and values. Drawing from prior motivational interventions grounded in expectancy-value theory described above as well as the broader literature on supports for self-efficacy (e.g., Bandura, 1986; Usher, 2009), intrinsic motivation (Reeve, Deci, & Ryan, 2004; Ryan & Deci, 2000), and interest (Pugh, Linnenbrink-Garcia, Perez, & Phillips, 2015), as well as more comprehensive theoretical accounts for supporting motivation (Linnenbrink-Garcia et al., 2016; Pintrich, 2003; Turner, Warzon, & Christensen, 2011), we identified five design principles that guided our motivational intervention: 1) inclusion of real-world challenging tasks, 2) provision of choice surrounding academic tasks, 3) encouragement of active involvement, 4) support for feelings of belonging, and 5) use of effort-based evaluation (see Table 1).
Table 1.
Design Principles Supporting Motivation in Summer Enrichment
| Design Principle | Form of Motivation Supported |
Enactment in Summer Program |
|---|---|---|
| (1) Real-world challenging task | Self-Efficacy Task value | Course content focused on pharmacology, emphasis on drugs to treat common diseases |
| (2) Provision of choice | Task value | Student-design inquiry lab experiments, problem-based learning activities, research proposals |
| (3) Active involvement | Task value | Lab experiments, small group activities, research proposals |
| (4) Support for belonging | Self-Efficacy Task value | Student housing, meals, group work, formal and informal interactions with students and instructors |
| (5) Effort-based evaluation | Self-Efficacy | No formal evaluation, emphasis on mastery experiences, formative feedback on research proposal |
Specifically, drawing on the success of psychological (e.g. utility value interventions, Harackiewicz et al., 2016) and instructional (e.g., Feng & Tuan, 2005) interventions targeting relevance and the use of real-world challenging problems to support both situational and individual interest (e.g., Dohn, Madsen, & Malte, 2009; Linnenbrink-Garcia, Patall, & Messersmith, 2013), the inclusion of real-world challenging tasks (design principle 1) is included to support heightened task value. Moreover, the “challenge” component of this principle has the added benefit of also supporting competence beliefs, as success on challenging tasks is a critical element of the mastery experiences linked to supports for self-efficacy (Bandura, 1986). The second (provision of choice) and third (active involvement) design principles should also support task value. Provision of choice, which ties directly to autonomy support, is supported by an extensive body of empirical research linking autonomy support to interest development and intrinsic motivation (e.g., Palmer, 2009; Patall, Cooper, & Wynn, 2010) and thus should also help to support task value, a motivational construct closely related to interest (Wigfield & Cambria, 2010). Active involvement draws primarily from the research on supporting interest development whereby various classroom activities that encourage students’ active engagement (e.g., working in groups, doing hands-on activities, engaging in group discussions) support students’ feelings of enjoyment and personal connection to a domain (e.g., Dohn et al. 2009; Renninger & Hidi, 2002), which also connect to the interest and attainment components of task value.
The fourth principle, support feelings of belonging, also draws from research on interest development and intrinsic motivation, where providing personal connections and feelings of relatedness are associated with heightened interest (Linnenbrink-Garcia et al., 2013; Renninger & Hidi, 2002) and intrinsic motivation (Reeve et al., 2004). Moreover, a sense of belonging may also strengthen the messages related to self-efficacy, such that social persuasions and positive feedback provided by instructors or peers may take on added value when students’ feeling personally connected to the individual providing feedback. Similarly, feelings of belonging may strengthen the effect of vicarious experiences on self-efficacy such that students may view other students and instructors in the program as more similar to themselves if they have a heightened sense of belonging and thus would benefit more from observing the success of similar others (Bandura, 1986). Finally, effort-based evaluation was included to support self-efficacy. When students are evaluated based on their progress and effort, this creates enhanced opportunities for mastery and success thereby supporting feelings of self-efficacy (Bandura, 1986).
Current Study
The current study utilized these five motivational design principles to create a multi-faceted motivational intervention embedded within a summer enrichment program for undergraduates between their first and second years of college (see Method for a detailed description of the enactment of the design principles in the program). We then evaluated the effectiveness of the program in supporting science self-efficacy and task value as well as subsequent science choices (science courses completed), achievement (science GPA), and science-research career intentions. To this end, we compared summer program participants to a propensity score matched no-treatment comparison group on science motivation eight months after participating in the summer enrichment program and on science persistence variables at the end of the third year of college. Importantly, we sought to examine how an intervention based on these five motivational principles functioned as a package to support motivation and persistence, rather than seeking to isolate the effects of each component since we view this combination of supports for motivation in a more holistic, synergistic way (see Guthrie, Wigfield, & VonSecker, 2000 for a similar approach). While motivational interventions that target a specific motivational mechanism are important (e.g., utility-value interventions), we argue it is also important to understand the motivational mechanisms by which a learning environment designed to holistically support motivation functions given our view that educational contexts and student motivation are complex, interrelated dynamic systems (see also Kaplan & Garner, 2017; Linnenbrink-Garcia & Wormington, 2017; Skinner, Kindermann, Connell & Wellborn, 2009; Turner & Patrick, 2008).
We targeted our intervention between the first and second year of college because this is a critical time during which students reflect upon their college experiences and begin to more carefully consider their planned major. Moreover, at this time, students had completed at least one of the required introductory chemistry courses for natural sciences major, which historically are viewed as “gate-keeper” courses that may make students question whether pursuing a degree in science is an achievable and worthwhile goal. As such, targeting interventions during these early years may be particularly important (Cromley, Perez, & Kaplan, 2016). We situated our intervention within the context of a summer program rather than a more traditional course for several reasons. First, designing our own program allowed us to select a topic (pharmacology) that helped to highlight the real-world relevance of basic topics in chemistry and biology, thereby helping students to see the connection of what they were learning in their introductory courses to possible real-world applications or broader career goals. Undergraduates would not typically be introduced to pharmacology until they take more advanced courses. Second, by designing a two-week full day program, we were able to include more opportunities for elongated experimentation, thus providing students with more autonomy in designing and implementing experiments and allowing them to more directly tackle real-world experiences. This would not typically be possible in an undergraduate lab course. Third, the more informal nature of the program provided unique opportunities to foster feelings of belonging and relatedness, both among students and with program staff. Fourth, the more informal nature of the program meant that we could eliminate formal evaluation, placing a greater emphasis on mastery experiences. Notably, the stand-alone summer program afforded us the freedom to design and implement the course in a manner fully consistent with our design principles, something that would have been challenging in the context of early undergraduate gateway science courses, which often rely on traditional lectures and pre-determined (“cookbook style”) lab assignments. Finally, prior research suggests that enrichment programs may be particularly useful for supporting student motivation and STEM persistence (e.g., Maton et al., 2009; Russell et al., 2007; Villarejo et al., 2008), suggesting that an enrichment experience may be an ideal context in which to embed our motivational intervention.
With respect to outcomes, we included both motivational outcomes and science persistence outcomes in order to evaluate the effectiveness of the summer program and the psychological mechanisms through which it functions. In particular, if a science-focused summer enrichment program can support students’ science motivation, it has the potential to change not only their immediate engagement, but their longer-term trajectories. This may be critically important, as students have a variety of choices about potential careers to pursue and enhancing motivation may be even more important than any immediate impacts on achievement, for example. The consideration of students’ science course-taking and GPA in their third year is an important indicator of whether they are actually enrolling in and performing well in science courses as part of their advanced coursework, which students typically begin taking in their third year of college at this institution. Furthermore, students’ completion of such courses is an indicator of their continuation in a science degree program and their success in these courses is key for pursuing careers in science after college. Thus, these two distal outcomes provide real-world indicators of science engagement. We also assessed students’ intentions to pursue a research career in science to provide an indicator of future plans beyond college. By including science persistence variables assessed almost two years after the intervention, we were able to examine whether there were any direct effects of the intervention on science persistence outcomes in the third year as well as whether there were indirect effects on these outcomes via science motivation in the second year. Thus, our longitudinal approach enabled us to test a longitudinal mediation model, which meets the more stringent requirements of the predictor, mediator, and outcome variables being measured at distinct time points (Maxwell, Cole, & Mitchell, 2011).
We evaluated two primary research questions: (1) Does participation in the summer enrichment program support students’ science motivation (self-efficacy and task value) 8 months after completing the summer program and science persistence (science research-career intentions, science course completion, science GPA) approximately 20 months after completing the summer program? and (2) Are there indirect effects of the summer enrichment program on distal indicators of science persistence via changes in science motivation? For both questions, we compared the summer enrichment program participants to a propensity score matched comparison group.
Given the utilization of our five motivational design principles in developing the summer program (see above), we hypothesized that the summer enrichment program would positively predict college students’ science task value, science self-efficacy, and science persistence (third-year science GPA, third-year science courses completed, third-year intentions to pursue a research career in science; RQ1). With respect to indirect effects (RQ2), we hypothesized that positive effects of participating in the summer enrichment program on student motivation would in turn enhance the science persistence outcomes. That is, by effectively supporting student motivation through the summer enrichment program, students would experience broader benefits in terms of their science persistence. Drawing from prior research on the unique effects of expectancies and task value on achievement and choice behaviors, respectively (Wigfield & Cambria, 2010), we hypothesized that science self-efficacy would be the strongest predictor of science GPA, but would also relate positively to science course completion and science career intentions. In contrast, we hypothesized that task value would be the strongest predictor of science course completion and career intentions but would also relate positively to science GPA. Thus, we hypothesized that the summer enrichment program would have a significant positive indirect effect on all three indicators of science persistence, with the strongest indirect effects for science GPA through science self-efficacy and the strongest indirect effects for science course completion and career intentions through task value.
Although not part of our primary focus, we also examined whether the summer enrichment program was differentially beneficial for women and for students with relatively lower science achievement during their first year in college. As noted previously, there is a concern about the dearth of women pursuing careers in STEM (Chen, 2013; National Science Foundation, 2015; PCAST, 2012), although this is less pronounced in scientific fields that are not focused on mathematics such as biomedical sciences (Ceci, Ginther, Kahn, & Williams, 2014). Moreover, there is some evidence that males and females may differ in their levels of perceived competence or value by domain (although many of these differences become smaller with age; for reviews see Meece, Glienke, & Burg, 2006; Wang & Degol, 2013). Thus, if female students have lower perceived competence and value, there may be a greater possibility of the interventions acting upon these beliefs as there is more potential space for growth. Thus, we hypothesized that, if gender differences were to exist, the intervention would be more effective among female than male students.
Drawing from prior research suggesting that motivational interventions may be particularly beneficial for students with lower perceived competence (e.g., Hulleman & Harackiewicz, 2009) or who are at greater risk for failing (e.g., Paunesku et al., 2015; Yeager et al., 2016), we also examined whether the summer enrichment program was more beneficial for students with lower science achievement in their first year in college. We hypothesized that if differences were to occur, the summer enrichment program would be especially beneficial for supporting science motivation and science persistence for students with lower achievement in science courses during their first year in college.
The current study contributes to both theory and practice in several key ways. First, the current push by some in motivation intervention research is to develop targeted interventions focused on a particular form of motivation (Harackiewicz & Priniski, 2018; Walton, 2014). Yet, as a field, we know that multiple elements of motivation are key for supporting students’ engagement and learning (e.g., Pintrich, 2003; Turner et al., 2011). Thus, investigating interventions that target multiple types of motivation represents an important advance in our understanding of how to support student motivation more holistically at the undergraduate level. Moreover, a clear benefit of embedding supports for motivation within an undergraduate enrichment experience is that it can provide a useful framework for others interested in implementing similar design principles with existing or new programs. Second, few motivational intervention studies consider the impact of the program on both the hypothesized motivational mechanism (in this case self-efficacy and task value) as well as a distal indicators of science persistence (but see Hulleman, Kosovich, Barron, & Daniel, 2017 for test of mediation within a semester). Our inclusion of both psychological mediators and persistence outcomes, assessed over multiple years, enables us to clearly test the hypothesized motivational mechanisms on more distal outcomes using longitudinal mediation models. Thus, our study is uniquely positioned to address questions of whether multi-faceted interventions can have a lasting impact on supporting student motivation and their intentions to pursue a science-related career. Finally, it is worth noting that motivational researchers have struggled to effectively translate motivational theory into practice (Linnenbrink-Garcia et al., 2016; Pintrich, 2003). Thus, the type of theoretically-derived field-based implementation research employed in the current study is critical for assessing whether theoretically-derived design principles are effective in supporting student motivation and subsequent outcomes.
Method
Participants
The sample for this study was drawn from a large, multi-year longitudinal project, which included 2,532 first-year undergraduate students enrolled in gateway chemistry courses at a highly selective university in the southeastern United States. Participants for the current study were from a propensity score matched sample (N = 587; see Propensity Score Matching section below) drawn from the first three cohorts of this larger project.
The propensity score matched sample for this study was 66.3% female and was 24.4% White, 45.8% Asian, 14.5% African American, 8.0% Latino, and 7.3% multi-racial/other. Of the 587 participants, 186 students participated in the summer enrichment program and 4011 students were in a no-treatment comparison group. The final comparison sample and summer program sample were similar to each other with respect to demographics (see Table 2).
Table 2.
Demographic Breakdown of Summer Enrichment Program Sample and Propensity Score Matched Comparison Sample
| Summer enrichment Program | Comparison Sample | |
|---|---|---|
| n | 186 | 401 |
| % Female | 66.1 | 66.3 |
| % Asian | 46.2 | 45.6 |
| % African American | 15.1 | 14.2 |
| % Hispanic/Latino | 7.5 | 8.2 |
| % White | 23.1 | 24.9 |
| % Other | 8.1 | 6.9 |
| M (SD) Age | 18.12 (.41) | 18.10 (.32) |
| Mother’s Highest Education (Mode) | College Degree | College Degree |
| Father’s Highest Education (Mode) | Doctorate/Professional Degree | Doctorate/Professional Degree |
| Annual Family Income (Mode) | $100,000 – $149,900 | $250,00+ |
Note. Mother/Father Education Options: 1 – grade school or less, 2 – high school or GED; 3 – college degree; 4 – master’s degree; 5 – Doctorate or Professional Degree (e.g., PhD, MD, JD); Income Options: 1–below $25,000; 2–$25,000–$49,999; 3–$50,000–$74,999; 4–$75,000–$99,999; 5–$100,000–$149,999; 6–$150,000–$199,999; 7–$200,000–$249,999; 8– $250,000+
Procedures
As part of an on-going longitudinal study, we administered baseline surveys in first-year undergraduate chemistry courses each fall semester between 2010 and 2012.2 Students were compensated $10 for completing the survey, with 73.3% of eligible students participating in the baseline survey. We used the baseline variables for the propensity score matching procedures.
In each spring semester between 2011 to 2013, we recruited first-year undergraduates in the College of Arts and Sciences and the School of Engineering to apply to a summer enrichment program (see detailed description of summer program below), which took place during the summer between students’ first and second years of college. The research team reviewed the applications and accepted nearly all students who applied to the program each summer.3 An average of 64 students (cohort 1 = 58, cohort 2 = 71, cohort 3 = 64) participated in the program each summer across the three cohorts included in this study.
In the spring of participants’ second year (fourth semester, 2012 to 2014) and third year (sixth semester, 2013 to 2015), 8 months and 20 months after the summer enrichment program, we invited all summer program participants and a sub-set of potential no-treatment comparison group participants to complete a follow-up survey. All students were recruited via email and completed the survey on-line. Students were compensated $10/survey for each follow-up survey they completed. Comparison group participants were randomly selected (blocked by race/ethnicity and gender to match the participants in the summer enrichment program) from the larger group of participants who completed the initial baseline survey. In total, we invited 1,037 potential comparison group participants to complete the follow-up survey across all three cohorts (approximately 346 per cohort). We intentionally invited a large number of potential comparison group participants in order to have a larger sample on which to use propensity score matching to create a matched comparison group. The response rate on the follow-up survey for the comparison group sample was 49% and 44%, respectively for surveys completed in their second and third years; 91% and 82% of summer program students completed the follow-up survey in their second and third years, respectively.
Summer Enrichment Program
The summer enrichment program was a two-week instructional program during which students learned about fundamental concepts in pharmacology (week 1) and drug treatments for four specific diseases (week 2). Students attended the program for 7 hours/day for 10 days. The majority of the course involved inquiry-based and active-learning activities, with less than one hour/day devoted to lecture. Each day was organized around a specific theme using topics relevant to college students (e.g., how drugs are eliminated from the body). For a final project, students chose a topic of interest and, with the guidance of the instructors, worked to design a hypothetical research study based on their topic of interest over the two-week period. The project culminated in a conference style poster presentation on the final day.4
Instructors for the course were postdoctoral fellows and doctoral students in the basic sciences (e.g., pharmacology, biology), most of whom had limited prior teaching experience. The program directors (faculty in pharmacology and psychology) delivered professional development to the instructional staff during two full-day workshops preceding the summer course. The majority of the workshops focused on teaching pedagogical techniques, including delivering effective lectures and serving as facilitators in small group, problem-based learning, and laboratory activities. Across the two days, approximately one hour was devoted to providing instructors with an overview of student motivation, introducing the five motivational design principles on which the summer program was developed, and giving tips for supporting student motivation in line with these design principles. More details regarding the instructor training, including sample materials, are available in Godin et al. 2015.
The summer program was developed to align with five motivational design principles described previously (see Table 1). Specifically, the first design principle, using real world challenging tasks, was incorporated into the selection of pharmacology as the subject matter. The summer program centered on the real-world application of basic biology and chemistry principles, with a specific focus on the use of drugs to treat common diseases (e.g., cancer).
The second and third design principles, provision of choice (i.e., autonomy support) and encouragement of active involvement, were also key underlying themes in the instructional design of the summer program. For instance, we included four inquiry-guided lab activities focused on one of four drugs (aspirin, caffeine, tobacco, alcohol). Students had the opportunity to design the experiments as a group, with guidance from instructors, rather than following a set of pre-determined doses and procedures, as is often the case in undergraduate laboratory courses. Other activities that allowed for active engagement included problem-based learning activities and time to work on preparing the final research proposal, which could be on any topic in pharmacology. These activities provided numerous opportunities for students to be actively, rather than passively, involved in their own learning. Moreover, the predominant use of active learning and open-guided inquiry supports students’ autonomy, as students are key decision makers in how to proceed with the learning activities.
We targeted our fourth design principle, support for feelings of belonging, by providing numerous opportunities for social interaction among students and with course instructors. For instance, students were housed in adjacent rooms in a single residence hall during the program and ate breakfast and lunch together each day, affording opportunities for informal interactions among students throughout the program. Additionally, there were numerous opportunities for small group work. Instructors were also encouraged to interact with students during free times (e.g., lunch, breakfast) and shared their pathways into graduate school with students.
Our final motivational design principle, use of effort-based evaluation, was focused on providing students with opportunities for growth in their learning and understanding. Towards this end, there were no formal grades; rather, we created numerous opportunities for mastery experiences and sought to reduce overt social comparison. Students received informal evaluation about the quality of their work and their effort in small groups. Additionally, instructors evaluated students’ research proposals through oral and written, ungraded, feedback throughout the development process; students also received written formative feedback after the research proposal poster session.
To assure fidelity of implementation of the enrichment program, a member of the research team with expertise in motivation and a member of the team with expertise in the pharmacology content observed most activities throughout each day for the entire program. At the end of each day, research team members met with the instructors to debrief. As needed, they provided the instructors with feedback about whether the instructional practices they were enacting aligned with the five motivational design principles and provided suggestions on how to better align their instruction. Research team members also provided more general suggestions to improve instructors’ overall pedagogy. This oversight of the program by the research team allowed for a strong fidelity of implementation of the design principles and content.
Measures
Several measures assessing motivational and career-related beliefs in science were included as part of the larger longitudinal study. For the current study, we focused on measures of students’ self-efficacy in science, task value in science, and intentions to pursue a science-research career. All items were assessed on a 5-point Likert scale except for the career intentions survey, which was on a 10-point scale. As part of the baseline survey, we queried students about their research experiences in their first year, which was used as a control variable in our models. A demographics questionnaire was also included at the end of the baseline survey. Finally, we collected science achievement data and science course completion data from the university institutional research office. We describe each of the measures included in our primary analyses in further detail below. Table 3 includes the reliability coefficients for the follow-up measures.
Table 3.
Descriptive Statistics Estimates for Observed Variables and Internal Consistency Reliability
| Variable | n | M | SD | Range | Reliability |
|---|---|---|---|---|---|
| Interest Value | 567 | 4.17 | .66 | 1.00 – 5.00 | .91 |
| Attainment Valuea | 566 | 3.94 | .71 | 1.00 – 5.00 | .80 |
| Utility Valuea | 565 | 4.22 | .60 | 1.00 – 5.00 | .81 |
| Self-Efficacy | 566 | 3.75 | .68 | 1.00 – 5.00 | .85 |
| Career Intentions | 425 | 5.25 | 2.92 | 1.00 – 10.00 | – |
| Science GPAb | 415 | 3.21 | .76 | 0.00 – 4.00 | – |
| Science Courses | 581 | 2.83 | 2.53 | 0.00 – 9.00 | – |
Note.. GPA = grade point average; Career Intentions = 3rd year science-research career intentions; Science GPA = 3rd year science GPA; Science Courses = 3rd year Science Courses Completed. All motivation variables were assessed at the first follow-up survey (assessed 8 months post enrichment program). Internal consistency reliability coefficients are Raykov’s Rho (Raykov, 2009) coefficients of latent variables calculated in Mplus; GPA is on a 4.0 scale.
Statistics are reported for the revised scales based on the CFA analyses.
Science GPA was only calculated for students who were enrolled in a graded science course during their third year.
Task value in science
Science task value was measured using 15 items adapted from Conley (2012). Items targeted three sub-components including attainment value (e.g., It’s important for me to be a person who reasons scientifically; 5 items), utility value (e.g., Science will be useful for me later in life; 5 items), and interest value (e.g., I enjoy doing science; 5 items). The full list of items is presented in the Appendix.
Science self-efficacy
Science self-efficacy was measured using a six-item scale (Estrada et al., 2011, see Appendix), which was designed to measure students’ perceptions of their ability to successfully execute a variety of scientific tasks. An example item read, “I am confident that I can generate a research question to answer.”
Science-research career intentions
We used a single item to measure students’ intentions to pursue a career in science research (Schultz et al., 2011), which read, “To what extent do you intend to pursue a research-related career in science?” This item used a Likert-type response scale ranging from 1 (definitely will not) to 10 (definitely will) and was administered in the third year follow up survey (i.e., 6th semester).
Science GPA and science course completion
In the summer following participants’ third year, we requested record data for participants in the study from the institutional research office including course enrollment data and grades for courses taken in participants’ first three years. For science GPA, we calculated the students’ GPA across all graded science courses completed in the students’ third year. We also calculated students’ first-year science GPA (Year 1 science GPA) for use in our ancillary interaction analyses.
For science course completion, we added the total number of science courses completed in the participants’ third year. We included all courses that were completed for credit regardless of whether the course grade was included in the GPA calculation (e.g., courses taken as pass/fail were counted as completed if the student passed the course).
Data Analyses
Propensity score matching
Because it was not possible to randomly assign students to participate in the supplemental summer program, we used propensity score matching (Rosenbaum & Rubin, 1983) to create a closely matched comparison group using data available from the baseline survey. While propensity score matching is not equivalent to random assignment to treatment groups, it allows the researcher to account for selection effects into a treatment group by statistically equating the treatment group and a comparison group using initial variables that predict participation in the treatment (Rosenbaum & Rubin, 1983). Specifically, a multivariable logistic regression was performed based on whether a student participated in the summer program (Treatment=1) or not (Comparison=0) using demographic variables (e.g., gender, URM status, race/ethnicity, age, native/non-native English speaker status, mother’s and father’s highest level of education, annual household income), ability (SAT score), indicators of initial interest/participation in STEM fields during the first year of undergraduate studies (premed student status, 1st year STEM course GPA, 1st year STEM course completion, baseline science-research career intentions, participation in research during the academic year, participation in research during the previous summer, science-related internship during the previous summer), and initial indicators of motivation from the first-year baseline survey as predictors of treatment status.
We used a stratification approach to propensity score matching whereby the sample is blocked into strata based on the propensity scores. Thus, both treatment and comparison group participants within a single stratum contain similar observations (based on the propensity score). The stratification approach accounts for the natural imbalance in the eligible students in the treatment and comparison groups (Stuart, 2010; see Results section for a description of the final sample retained using this approach).
Finally, to evaluate the effectiveness of the propensity score matching approach for equating the treatment and comparison groups, we conducted a sensitivity analysis with regard to each of the baseline characteristics to examine initial differences between the summer program and comparison groups after propensity score matching. Results of these analyses are reported in the Results section below.
Primary analyses
We employed confirmatory factor analysis (CFA) using Mplus (v. 7.3) to analyze the factor structure of the latent motivation variables. Since we did not have specific hypotheses related to how the intervention would affect each form of task value and since the task value latent variables were highly correlated with each other, we modeled task value as a second order latent variable. We used multiple-indicator multiple-cause (MIMIC) structural equation modeling (Kline, 2011; Thompson & Green, 2013) to answer our primary research questions (see Figures 1 and 2). In these models, the summer enrichment program was included as a dichotomous variable and the effect of this variable on the outcomes is interpreted as the mean difference between the comparison group (coded 0) and the summer enrichment group (coded 1). We tested for measurement invariance between the summer enrichment and comparison groups in our CFA models since measurement invariance across groups is an assumption in MIMIC models. Following standard accepted procedures for testing weak, strong, and strict invariance across groups (Kline 2011; Thompson & Green, 2013), we used Chen’s (2007) guidelines for comparing nested models.
Figure 1.
MIMIC model examining the direct effects of participating in the summer enrichment program on science task value and self-efficacy, science GPA, science courses completed, and science-research career intentions. The first coefficients in a path are standardized coefficients, which are Cohen’s d effect sizes. Unstandardized coefficients are in parentheses. Task Value is a second-order latent variable predicting first order latent variables for attainment value, utility value, and interest value. To simplify the figure, the measurement portion of the model is not depicted. GPA = grade point average; Career Intentions = science-research career intentions; †p < .10, *p < .05, **p < .01, *** p < .001.
Figure 2.
MIMIC model examining the indirect effects of participating in the summer enrichment program on science GPA, science courses completed, and science-research career intentions via motivation. The first coefficient in a path is standardized, and coefficients in parentheses are unstandardized coefficients. Standardized coefficients for direct effects of the summer enrichment program on the outcomes are Cohen’s d effect sizes. Task Value is a second-order latent variable predicting first order latent variables for attainment value, utility value, and interest value. To simplify the figure, the measurement portion of the model is not depicted. GPA = grade point average; Career Intentions = science-research career intentions; †p < .10, *p < .05, **p < .01, *** p < .001.
We used robust maximum likelihood estimation (MLR) in all models and model fit was assessed using the model χ2, Confirmatory Fit Index (CFI), Root Mean Square Error of Approximation (RMSEA), and Standardized Root Mean Square Residual (SRMR). Cutoff values of close to ≥ .95 were used for CFI, ≤ .06 for RMSEA, and ≤ .10 for SRMR to determine acceptable model fit (Hu & Bentler, 1999). We accounted for the stratification within the data by using the STRATIFICATION and COMPLEX commands in MPlus (Muthén & Muthén, 2012). Of particular interest for this study was the indirect effects of the summer enrichment program on the science persistence variables via motivation (see Figure 2). We calculated the specific and total indirect effects (e.g., mediated through all the motivation variables simultaneously) using Mplus’s MODEL INDIRECT command (Muthén & Muthén, 2012). Finally, we used Full Information Maximum Likelihood (FIML) to account for missing data in the analyses.
Results
Propensity Score Matching
Using stratification, we retained 587 participants (186 from the summer program and 401 from the comparison).5 The retained participants were divided into 5 quintiles based on their propensity scores, creating 5 strata. Sensitivity analyses revealed that across all 5 strata, there was only a statistically significant difference between the summer program and comparison participants for the first stratum with respect to participation in academic research during the first year of college (p = .040) and the third stratum with respect to native English speakers (p = .042). Participants in the summer enrichment program group in stratum 1 were more likely to have participated in a research experience during their first year in college (18.2%) compared to those in the comparison group (4.5%). Additionally, summer program participants in stratum 3 were more likely to be native English speakers (91% of summer program participants versus 77% of the comparison group). Caution should be used in interpreting these differences, however, given that relatively few individuals within the whole stratified sample participated in research during their first year in college (12.4%) or were non-native English speakers (23.2%), making comparisons between treatment groups within each stratum more challenging. There were no other significant differences in any of the 5 strata for the remaining 20 baseline variables. Thus, the sensitivity analyses suggest that matching by strata resulted in balance between the two groups (aside from the minor differences in research experience and English as a first language within two of the five strata).
Descriptive Statistics and Correlations
Descriptive statistics for observed variables and reliability coefficients are reported in Table 3. The correlation matrix for the latent variables is presented in Table 4. Overall, the correlation results were consistent with what would be expected from prior research and theory. Furthermore, the summer enrichment program was significantly positively correlated with all variables except third-year science GPA.
Table 4.
Correlation Matrix for Latent Variables
| 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | |
|---|---|---|---|---|---|---|---|---|---|
| 1. Interest Value | – | ||||||||
| 2. Attain Value | .77 | – | |||||||
| 3. Utility Value | .74 | .80 | – | ||||||
| 4. Self-efficacy | .40 | .43 | .41 | – | |||||
| 5. Task Value | .85 | .91 | .88 | .47 | – | ||||
| 6. Career Intentions | .37 | .39 | .38 | .20 | .43 | – | |||
| 7. Science Courses | .23 | .25 | .24 | .15 | .27 | .23 | – | ||
| 8. Science GPA | .11 | .12 | .11 | −.02 | .13 | .09 | .27 | – | |
| 9. Research Exper. | .11 | .12 | .11 | .10 | .13 | .07 | .11 | .14 | – |
| 10. Summer Program | .14 | .14 | .14 | .13 | .16 | .11 | .12 | −.01 | .00 |
Note. All correlations > |.09| are significant at p < .01. Since the summer enrichment program is a dichotomous variable, point biserial correlations between the summer program and the other variables are reported. GPA = grade point average; Career Intentions = 3rd year science-research career intentions; Science GPA = 3rd year science GPA; Courses Completed = 3rd year Science Courses Completed; Task Value is a second-order latent variable predicting first order latent variables for attainment value, utility value, and interest value; Research Exper. = Research experiences in the first year.
Confirmatory Factor Analysis
In the CFA models, we first tested a first-order latent variable model in which latent variables for each of the four expectancy-value variables (self-efficacy, interest value, attainment value, utility value) predicted the relevant indicator variables for both the comparison and summer enrichment program groups. While fit statistics were acceptable in these models, modification indices suggested that model fit would be significantly improved by specifying two utility value items and one attainment value item on other latent variables not supported by theory (expected parameter change > .25). Therefore, we dropped the problematic items from our analyses and re-ran the models (see Appendix for final measures). Given the high correlation between the task values latent variables (latent correlations were all >.70), we next modeled a second-order latent variable for task value. Finally, we tested for measurement invariance across the treatment and control groups. Results of the measurement invariance tests suggested partial strict invariance of the motivation latent variables across the comparison and summer enrichment program participants (strong invariance across groups was also established). The fit of the final CFA model was good; χ2(315) = 531.393, RMSEA = .049, CFI = .954, SRMR = .093 (see Table S1 in supplemental materials for further details about measurement invariance tests). Therefore, we were able to proceed with analyzing our data using the MIMIC model approach.
Primary Analyses: MIMIC Model Results
Figure 1 presents the direct effects of the summer enrichment program on the motivation and science persistence variables. Figure 2 presents the indirect effects model of the summer enrichment program on the science persistence variables via motivation. We controlled for first year research experiences since there was a small difference between the comparison group and summer enrichment program group in one of the strata. While there was also a small difference in students who spoke English as their first language in one of the strata, this variable was not related to any other variables in the model so was eliminated from our final models. Fit statistics for the final MIMIC models suggested a good model fit to the data; direct effects model: χ2(219) = 523.659, RMSEA = .049, CFI = .942, SRMR = .081; indirect effects model: χ2(213) = 444.570, RMSEA = .043, CFI = .956, SRMR = .037. We report results specific to each of our research questions below.
RQ1: Effects of summer enrichment program on motivation and science persistence
Our first research question focused on the direct effects of the summer enrichment program on both motivation (assessed eight months later) and science persistence (assessed 20 months later). For these analyses, the unstandardized direct effects (presented in parentheses in Figure 1) of the summer enrichment program on the outcome variables are interpreted as the mean difference between the summer program group and the comparison group on these dependent variables.
With respect to motivation, results indicated that participating in the summer enrichment program yielded the expected positive effects on students’ science motivation relative to a no-treatment comparison group. Specifically, participation in the summer program was associated with significantly higher latent mean task value and self-efficacy, controlling for first-year research experience (see Figure 1).
In terms of science persistence, participation in the summer enrichment program between students’ first and second years had a direct effect on third-year career intentions, controlling for first-year research experiences. Specifically, summer enrichment program participants reported higher mean levels of intentions to pursue a science-research career compared to the no-treatment comparison group. Similarly, there was a direct effect on completion of science courses in the third year such that enrichment program participants completed, on average, more science courses in their third year relative to the comparison group. However, there was no statistically significant difference between summer enrichment students and comparison students on science GPA. Thus, students who participated in the summer enrichment program reported higher mean science-research career intentions and completed more science courses in their third year in college than the comparison group, but did not perform better in science courses.
RQ2: Indirect effects of the summer program on science persistence via motivation
Results indicated that the summer enrichment program had significant and positive indirect effects on two of the three science persistence outcomes (science course completion and science-research career intentions, see Figure 2 and Table 5). There were no significant total indirect effects of the summer enrichment program on science GPA, thus we do not discuss this outcome further.
Table 5.
Coefficients for Indirect Effects of the Summer Enrichment Program on Science Persistence via Follow-up Motivation
| Science Persistence Variables | |||
|---|---|---|---|
|
|
|||
| Effects Through | Science GPA1 | Science Courses Completed2 | Career Intentions3 |
| Total Indirect | .03 (.02) | .09 (.22**) | .14 (.42***) |
| Task Value | .06 (.05*) | .08 (.21**) | .15 (.43**) |
| Self-Efficacy | −.03 (−.02) | .01 (.01) | .00 (−.01) |
Note. Standardized coefficients (based on STDY standardization) are presented first; Unstandardized coefficients are in parentheses; GPA = grade point average; Career Intentions = 3rd year science-research career intentions; Science GPA = 3rd year science GPA; Courses Completed = 3rd year Science Courses Completed.
Scores for science GPA ranged from 0 – 4.
Scores for science courses completed ranged from 0 – 9.
Scores for career intentions ranged from 1–10.
p < .10.
p < .05.
p < .01.
p < .001.
The summer enrichment program had positive effects on science courses completed and science-research career intentions via task value. There were no indirect effects on persistence outcomes via self-efficacy. As noted above, students who participated in the enrichment program reported higher task value and higher self-efficacy than comparison students. However, task value, but not self-efficacy, was associated with stronger intentions to pursue a science research career and the completion of more science courses. Thus, the only potential pathway to science persistence outcomes was through task value, not self-efficacy.
Although the direct paths from the summer program to career intentions and courses completed were both reduced in the indirect effects model, the direct effect of the summer program on courses completed remained statistically significant. Thus, there was a direct effect of the summer program on courses completed and an indirect effect via task value. For career intentions, there was only an indirect path through task value once the motivational variables were included as mediators in the model.
Ancillary Analyses
In addition to testing the main effects of participating in the summer enrichment program on students’ motivation and science persistence, we also considered whether the summer enrichment program functioned differently for males and females and based on first-year undergraduate science achievement.
Gender interaction analyses
To assess whether the program functioned similarly across gender, we tested a summer enrichment program × gender interaction on the motivation and the science persistence outcome variables. Specifically, we tested separate direct effects models for each of the variables (i.e., 2 motivation variables and 3 science persistence variables) and we included a summer enrichment × gender interaction variable in each of the models. First-year research experience was also included as a control variable in all models.
Results indicated a statistically significant summer enrichment program × gender interaction effect in one of the five models. Specifically, there was a statistically significant summer enrichment program × gender interaction on third-year science courses completed (b = −1.05, β = −.42, p = .027, see Figure 3).6 Contrary to our hypothesis, a simple slopes analysis indicated that there were statistically significant differences in science course completion between the comparison group and summer enrichment program for males (t = 3.46, p = .001) but not for females (t = 0.38, p = .705). Notably, both males and females in the summer enrichment program completed a similar number of science courses during their third year in college (t = −0.23, p = .822). However, in the comparison group, males completed fewer science courses than females; although, this difference was marginally significant (t = 1.83, p = .068). Thus, participating in the summer enrichment program was equally effective for both males and females in all outcomes except for the number of courses completed, where males’ course-taking in science received a boost.
Figure 3.
Gender × summer enrichment program interaction effect on 3rd year science course completion.
Interactions with first-year science GPA
Using the same method as described in the gender interaction analyses, we tested interactions between participating in the summer enrichment program and first-year science achievement to examine whether participating in the enrichment program was equally effective for students with different first-year achievement levels. Results indicated a significant effect of the interaction between the summer enrichment program and first-year science GPA on task value (b = −0.17, β = −.32, p = .032, see Figure 4). For students with a lower first-year GPA (1 SD below the mean of first-year GPA), a simple slopes analysis revealed there was a statistically significant difference in task value between students who participated in the summer enrichment program and those in the comparison group (t = 3.75, p < .001). Lower-performing summer enrichment program participants were 0.32 standard deviations higher on mean latent task value relative to lower-performing comparison group participants. However, such a difference was not found for higher performing students (t = 0.97, p = .335). These results suggest that participating in the summer enrichment program was particularly beneficial for supporting the task value of students whose science GPA was comparatively lower within this elite sample of students. There was no other significant enrichment program × prior achievement interactions.
Figure 4.
Prior achievement × Summer enrichment program interaction effects on task value.
Discussion
Our findings add to the existing literature suggesting that a motivational approach may be particularly useful for addressing the leaky STEM pipeline through science motivation. Specifically, our study contributes to the literature in two primary ways. First, our more holistic, synergistic approach to developing and implementing a motivational intervention is relatively unique, as there are only a few studies that take such an approach (e.g., Guthrie et al., 2000; Turner et al., 2011) despite calls from motivation researchers for such synergistic approaches in authentic learning environments (e.g., Turner & Patrick, 2008). Second, aside from Hernandez et al. (2013), studies focused on the relation of enrichment experiences to STEM persistence have not considered whether enrichment programs support persistence through support for psychological mechanisms. Third, much of the work on motivational interventions focuses on the effect of the intervention on STEM persistence, with only a few studies (e.g., Hulleman et al., 2017) testing the proposed motivational belief as a mediator. Thus, our approach to examining the relation of a multi-faceted motivational intervention to sustained motivation and, in turn, to STEM persistence provides unique insights into the potential psychological processes through which such as multi-faceted intervention may have sustained effects on STEM persistence. Below, we discuss our primary findings, noting both the alignment with prior research as well as these unique contributions and consider the implications of our findings for both practice and theory.
Efficacy of Undergraduate Enrichment Program for Supporting Motivation and STEM Persistence
Our findings provide evidence that it is possible to develop an undergraduate enrichment program that supports multiple forms of motivation (self-efficacy and task value) up to eight months after the program has ended, relative to a propensity score matched comparison group. This is critical, as prior research suggests that students’ motivation in science may be especially important for supporting STEM persistence (Wang & Degol, 2013) and prior attempts to alter undergraduates’ science motivation have not directly targeted multiple beliefs such as self-efficacy and values (e.g., Harackiewicz et al., 2016). Thus, our results provide evidence that undergraduate enrichment programs can be effectively designed to support multiple forms of student motivation, even eight months after the end of the intervention. Consistent with prior research noting that motivational interventions may be particularly effective for lower achieving students or those with lower perceived competence (e.g., Hulleman & Harackiewicz, 2009; Paunesku et al., 2015; Yeager et al., 2016), we also found evidence that the summer program was especially effective in supporting task value for students who had lower performance (even in this elite sample) relative to their peers in their first-year science courses (prior to participating in the enrichment program).
Additionally, students who participated in the summer program reported higher intentions to pursue a research career in science and took more science courses during the third year in college (close to two years after the program ended), relative to a propensity score matched comparison group. This pattern of results is in keeping with prior research highlighting the benefits of undergraduate enrichment experiences in supporting STEM persistence (e.g., Hernandez et al., 2013; Jones et al., 2010; Schultz et al., 2011). However, as noted in the introduction, the majority of prior research on undergraduate enrichment experiences is retrospective. There are only a few extant prospective studies (Hernandez et al., 2013; Jones et al., 2010; Schultz et al., 2011) and these examined research experiences, whereas we examined a 2-week enrichment program intentionally designed to boost self-efficacy and task value. Thus, our study provides critical empirical evidence that participating in a relatively short-term summer enrichment program can help to support science course taking two years later. This type of longitudinal research is critical to providing clear evidence about the types of institutional supports that are needed to effectively support STEM persistence.
The direct effects for third-year science course taking were qualified by a significant summer program by gender interaction. Males, but not females, who participated in the summer enrichment program took more science courses in their third year of college. This gender interaction appears to be driven by the relatively low science course taking in the comparison group for males. Males in the comparison group took about two science courses during their third year in college while females in the comparison group took about three courses. In contrast, both males and females in the summer enrichment program took about three science courses in their third year in college. This somewhat unexpected gender difference may be because we focused on science courses only, which include biology, neuroscience, and environmental studies courses. The gender gap in science is most pronounced in the physical sciences, mathematics, and engineering (Ceci et al., 2014). Thus, surprisingly, it appears that, at least in terms of direct effects, the summer enrichment program closed a gender gap for males in this context.
Notably, there were no significant direct effects of the summer program on science GPA. These results are not consistent with prior research suggesting that participating in undergraduate enrichment experiences such as undergraduate research was associated with higher achievement in biology (Jones et al., 2010). Part of the difficulty in predicting science GPA may have been due to the relatively low variability in GPA for this sample. While GPA is an important variable as it serves as a gatekeeper for continued study in science, the mean science GPA for our sample was well over a 3.0, suggesting that the majority of students in the sample are achieving at more than sufficient levels to continue their studies in science. These relatively high GPAs may be a unique feature of this particular context (e.g., elite undergraduate institution). Given this constraint, other indicators of persistence, such as continued science course taking and intentions to pursue research careers in science, may be better indicators of whether students do actually go on to pursue science careers.
Motivation as Mediator of Summer Program to STEM Persistence
As noted earlier, a critical contribution of the current study is that we considered science motivation as an underlying psychological mechanism for understanding how and why undergraduate enrichment experiences support science persistence using longitudinal mediation. We found that task value, but not self-efficacy, mediated the relation between participating in our summer enrichment program and two key indicators of science persistence (science research career intentions and science course taking) almost two years after the end of the summer program. This overall pattern of mediation highlights the potential value added when enrichment activities are specifically designed to support motivation and contrasts with prior research using utility-value interventions whereby utility value was not a significant mediator (Hulleman et al., 2017).
While the summer program was associated with higher self-efficacy eight months later, science self-efficacy did not significantly predict any of the indicators of science persistence. Given prior research highlighting the importance of self-efficacy in predicting achievement and of task value in predicting choice (Wigfield & Cambria, 2010), especially when value is also included in the model (Andersen & Ward, 2014), it is not surprising that self-efficacy was not a significant mediator in the relation between the summer program and career intentions and course taking. It is more surprising that science self-efficacy did not predict science achievement, especially in light of Hulleman and colleagues’ (2017) findings that the effect of a utility value intervention on final exams was mediated through expectancy for low-performing students. However, our findings may be due to the constraints in variability associated with the science GPA measure that we noted earlier. Nonetheless, self-efficacy is an important motivational variable in predicting a variety of achievement-related behaviors (Schunk & Pajares, 2005; Wigfield & Cambria, 2010). Thus, while we did not find significant effects of science self-efficacy on the persistence outcomes studied here, it is still important to note that program participants did have higher science self-efficacy eight months after the program ended. Importantly, there may be other benefits for science-related persistence and academic achievement that were not included in the current study, such as actual enrollment in a science graduate program. Overall, the results of the indirect effects suggest that, when it comes to course enrollment and career intentions, it may be most important to support task value-related beliefs for students in elite contexts such as in this study. However, it is likely important to also support the self-efficacy of students in more academically diverse contexts.
As such, our findings extend prior research by identifying specific psychological mechanisms (e.g., enhanced task value) that help to explain why undergraduate enrichment programs may support STEM persistence. Understanding the psychological mechanisms that explain why undergraduate enrichment programs support persistence is key, as knowledge of the specific psychological mechanisms through which these experiences function can aid in the design of more effective undergraduate enrichment experiences. Thus, the current study makes a unique contribution by identifying enhanced task value as at least one reason why undergraduate enrichment experiences are effective in supporting persistence.
Practical and Theoretical Implications
Finally, our study has both theoretical and practical importance by providing evidence that motivationally-designed enrichment programs or experiences can be an effective tool for addressing the leaky pipeline. From a practical perspective, our findings suggest that program designers could utilize the five design principles we identified (inclusion of real-world challenging tasks, provision of choice surrounding academic tasks, encouragement of active involvement, support for feelings of belonging, and use of effort-based evaluation) to design or change an existing undergraduate enrichment experience to support multiple forms of motivation. Our approach for supporting science motivation could also be implemented within the regular undergraduate STEM curriculum, thereby potentially reaching an even broader population of students. For instance, university professors could utilize these design principles for course design (syllabus, grading structure) as well as when developing classroom activities to support student learning. And, motivational researchers might work with an undergraduate department or unit to train faculty about how to implement these five motivational design principles within their undergraduate courses. Incorporating these design principles into courses or existing programs would cost very little and thus might be a cost-effective strategy for supporting STEM persistence. However, we also recognize that it is often difficult to change the structure of introductory science courses and enrichment programs may offer the most realistic opportunity to fully employ these design principles.
Theoretically, we need more research that takes the claims made by motivational researchers about how to support multiple forms of motivation in educational contexts and translates them into practice (Linnenbrink-Garcia et al., 2016; Pintrich, 2003). Our study provides evidence that such an approach is effective in terms of supporting multiple forms of student motivation and subsequent persistence in science. In conducting this work, we took a holistic, synergistic approach to supporting student motivation (see Guthrie et al., 2000 for a similar approach), with the goal of conducting research aligned with the view that educational contexts are complex, interrelated dynamic systems and that students’ motivation itself is a complex system (see Kaplan & Garner, 2017; Linnenbrink-Garcia & Wormington, 2017; Skinner et al., 2009; Turner & Patrick, 2008). As such, our goal was not to examine whether our approach to designing a summer enrichment program was better than other existing science enrichment programs, nor were we seeking to evaluate whether one specific motivational design principle was more effective than others to support motivation. Rather, we aimed to test the effectiveness of an enrichment program designed around a holistic set of motivational principles and to also examine the mechanisms by which such a program is effective. While we see value in careful experimental work that isolates a particular motivational mechanism, we contend that a more holistic approach to designing educational contexts may also be fruitful for designing educational contexts that support multiple forms of motivation. Thus, we argue that as we seek to translate theory to practice in ecologically-valid contexts, it is not always practical nor necessarily desirable to carefully test the individual components of one program versus another as the various components interact in the context and with the individual.
Limitations and Future Directions
There are several limitations to the current study that are worth noting when interpreting the findings. First, students were not randomly selected to participate in the summer enrichment program. We addressed this limitation by using propensity score matching to equate students on more than 20 baseline variables representing factors that would likely influence students’ decisions to participate in the summer enrichment program including demographic, initial interest/participation in science fields during the first year of undergraduate studies, and initial science motivation. However, while propensity score matching is an effective statistical tool for addressing selection effects to statistically equate control and treatment groups (Rosenbaum & Rubin, 1983), it is not a replacement for random assignment. For example, there may be variables that we did not collect (e.g., conscientiousness), and therefore were not included in the propensity score matching, that may have influenced students’ decisions to participate in the enrichment program. As such, causal claims regarding the effects of participating in the summer enrichment program on students’ science motivation and science persistence should be interpreted cautiously.
A second limitation is that students were only followed through the end of their third year at the university. Thus, we are not able to make claims regarding longer-term STEM persistence through college graduation and beyond. This sample of students is still being surveyed and we are continuing to collect academic record data. Future studies will be able to investigate the longer-term impact of our early college motivationally designed summer enrichment program. Nonetheless, these third-year indicators of persistence provide a reasonable window into the ultimate career path these students intend to take as they reflect upper-level course taking in science suggesting at least some degree of commitment to the field.
Third, recent advances in expectancy-value theory research highlight the importance of cost perceptions in predicting STEM persistence (Barron & Hulleman, 2015; Perez et al., 2014; Trautwein et al., 2013). Unfortunately, we did not have an adequate cost measure to include in this study. Future research, however, should more carefully consider whether undergraduate enrichment programs may be beneficial by not only enhancing expectancies and values, but also through a reduction in perceived costs associated with the pursuit of STEM careers. Future research might also consider whether motivationally-based undergraduate enrichment programs differentially support new distinct types of task value, such as differentiating between multiple types of attainment value (personal importance v. broad importance, Gaspard et al., 2015) or multiple types of utility value (short- versus long-term, Durik, Schechter, Noh, Rozek, & Harackiewicz, 2015).
Finally, we sampled participants from an elite university, which may limit the generalizability of our findings to other university settings. However, there are also several benefits to the population studied here with respect to addressing the leaky pipeline. First, our sample represents a group of highly qualified students. Thus, students in our sample should have the ability to pursue a science career. Second, our entire sample of participants (both the treatment and comparison groups) were enrolled in a first-year gateway science course, indicating at least some initial inclination to pursue a science-related career. Third, the university where this study was conducted uses a need-blind admissions process and is committed to enrolling a diverse student population (for instance only about 50% of the university undergraduate population is White). Together these sample characteristics allowed us to capture a group of diverse, highly qualified students at the start of the undergraduate science pipeline, thus providing important insights into science persistence among a population of students whom we would very much like to retain in the science pipeline.
Conclusion
The current study provides evidence of the efficacy of a motivationally-designed summer enrichment program for supporting undergraduates’ science self-efficacy and task value, and in turn, their science persistence through their third year in college. As expected, the summer enrichment program was positively associated with science self-efficacy and task value, assessed eight months after the end of the program, as well as the more distal outcomes of third-year intentions to pursue a science research career and third-year science course taking, both directly and indirectly via task value. Participation in the enrichment program was particularly beneficial for students with relatively lower first-year science GPA in terms of supporting task value, and for males in terms of third-year science course taking. Overall, the results highlight the potential positive impact of such a program for repairing the leaky STEM pipeline at the undergraduate level. These findings underscore the potential benefit of designing curricula and instruction to support multiple forms of student motivation, as doing so has the potential to not only shape motivational beliefs but also alter more distal science-career related beliefs and behaviors. As such, our approach provides a framework for undergraduate faculty interested in altering the learning environment through both structural and pedagogical elements to support science motivation and increase STEM persistence. It also provides critical evidence to motivational researchers that these theoretically-derived, but relatively untested, motivational design principles can be successfully implemented in authentic educational contexts to shape students’ motivation and persistence, thus helping to inform our understanding of how we might move forward in our design of educational contexts to support motivation.
Supplementary Material
Highlights.
Developed a motivationally supportive undergraduate enrichment program in science
The enrichment program supported undergraduates’ science motivation 8 months later
The enrichment program supported science persistence 20 months later
The enrichment program predicted science persistence indirectly via motivation
Acknowledgments
The research reported in this manuscript was supported by a grant from the National Institute of General Medical Sciences of the National Institutes of Health under award number 5R01GM094534. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health.
Appendix
Task Value (adapted from Conley, 2012)
Interest value
I enjoy the subject of science.
I enjoy doing science.
Science is exciting to me.
I am fascinated by science.
I like science.
Attainment value
It is important for me to be a person who reasons scientifically.
Thinking scientifically is an important part of who I am.*
Being someone who is good at science is important to me.
It is important for me to be someone who is good at solving problems that involve science.
Being good in science is an important part of who I am.
Utility value
Science concepts are valuable because they will help me in the future.
Science will be useful for me later in life.
Science is practical for me to know.*
Science helps me in my daily life outside of school.*
Being good in science will be important for my future (like when I get a job or go to graduate school).
*Indicates items dropped from the final measures based on CFA.
Science Self-Efficacy (Estrada et al., 2011)
I am confident I can…
Use technical science skills (use of tools, instruments, and/or techniques).
Generate a research question to answer.
Figure out what data/observations to collect and how to collect them.
Create explanations for the results of the study.
Use scientific literature and/or reports to guide research.
Develop theories (integrate and coordinate results from multiple studies).
Footnotes
Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final citable form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.
Earlier versions of the manuscript were presented at the annual meetings of the American Educational Research Association in April 2014 and April 2016.
We excluded 46 potential comparison group students prior to conducting propensity score matching because they had participated in other summer programs similar to the summer programs with similar goals to ours would likely confound the results of our study.
A small percentage of participants (8.43%) completed the baseline survey using an electronic survey at a later date rather than completing the paper survey in class.
Percent acceptance was 100%, 97.3%, 100% for Cohort 1, 2, and 3, respectively.
There were two within-program conditions included in the summer enrichment program: (1) a brief growth-mindset intervention and (2) participation in a fall research course in pharmacology, for which students received course credit. There were no differences on substantive measures between those who participated in either of these within-program conditions and those who did not participate. Thus, we did not differentiate among these conditions in our analyses.
The remaining participants (7 summer enrichment program, 37 comparison group) were dropped because they were outliers (i.e., there were no overlapping propensity scores between the treatment and comparison group participants).
βs for interaction effects are based on STDY standardization in Mplus (v.7.3).
References
- Andersen L, Ward TJ. Expectancy-value models for the STEM persistence plans of ninth-grade, high-ability students: A comparison between black, Hispanic, and white students. Science Education. 2014;98:216–242. doi: 10.1002/sce.21092. [DOI] [Google Scholar]
- Bandura A. Social foundations of thought and action: A cognitive social theory. Englewood Cliffs, NJ: Prentice Hall; 1986. [Google Scholar]
- Barron KE, Hulleman CS. Expectancy-value-cost model of motivation. In: Eccles JS, Salmelo-Aro K, editors. International Encyclopedia of Social and Behavioral Sciences 2nd Edition: Motivational Psychology. Amsterdam, Netherlands: Elsvier; 2015. pp. 261–271. [Google Scholar]
- Ceci SJ, Ginther DK, Kahn S, Williams WM. Women in academic science: A changing landscape. Psychological Science in the Public Interest. 2014;15:75–141. doi: 10.1177/1529100614541236. [DOI] [PubMed] [Google Scholar]
- Chen FF. Sensitivity of goodness of fit indexes to lack of measurement invariance. Structural Equation Modeling. 2007;14:464–504. doi: 10.1080/10705510701301834. [DOI] [Google Scholar]
- Chen X. STEM attrition: College students’ paths into and out of STEM fields. Washington, DC: National Center for Education Statistics, Institute of Education Sciences, U.S. Department of Education; 2013. [Google Scholar]
- Conley AM. Patterns of motivation beliefs: Combining achievement goal and expectancy-value perspectives. Journal of Educational Psychology. 2012;104:32–47. doi: 10.1037/a0026042. [DOI] [Google Scholar]
- Cromley JG, Perez T, Kaplan A. Undergraduate STEM achievement and retention: cognitive, motivational, and institutional factors and solutions. Policy Insights from the Behavioral and Brain Sciences. 2016;3:4–11. doi: 10.1177/2372732215622648. [DOI] [Google Scholar]
- Dohn NB, Madsen PT, Malte H. The situational interest of undergraduate students in zoophysiology. Advances in Physiology Education. 2009;33:196–201. doi: 10.1152/advan.00038.2009. [DOI] [PubMed] [Google Scholar]
- Durik AM, Shechter OG, Noh M, Rozek CS, Harackiewicz JM. What if I can’t? Success expectancies moderate the effects of utility value information on situational interest and performance. Motivation and Emotion. 2014;39:104–118. doi: 10.1007/s11031-014-9419-0. [DOI] [Google Scholar]
- Eccles JS, Adler TF, Futterman R, Goff SB, Kaczela CM, Meece JL. Expectancies, values, and academic behaviors. In: Spence JT, editor. Achievement and Achievement Motivations. San Francisco: Freeman; 1983. pp. 75–146. [Google Scholar]
- Estrada M, Woodcock A, Hernandez PR, Schultz P. Toward a model of social influence that explains minority student integration into the scientific community. Journal of Educational Psychology. 2011;103:206–222. doi: 10.1037/a0020743. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Feng S, Tuan H. Using ARCS model to promote 11th graders' motivation and achievement in learning about acids and bases. International Journal of Science and Mathematics Education. 2005;3:463–484. doi: 10.1007/s10763-004-6828-7. [DOI] [Google Scholar]
- Fry CL. Achieving systemic change: A sourcebook for advancing and funding undergraduate STEM education. Washington, DC: Association of American Colleges and Universities; 2014. Retrieved from http://www.aacu.org/publications. [Google Scholar]
- Gaspard H, Dicke A-L, Flunger B, Schreier B, Häfner I, Trautwein U, Nagengast B. More value through greater differentiation: Gender differences in value beliefs about math. Journal of Educational Psychology. 2015;107:663–677. doi: 10.1037/edu0000003. [DOI] [Google Scholar]
- Godin EA, Wormington SV, Perez T, Barger MM, Snyder KE, Richman LS, Schwartz-Bloom R, Linnenbrink-Garcia L. A pharmacology-based enrichment program for undergraduates promotes interest in science. CBE Life Sciences Education. 2015;14:1–12. doi: 10.1187/cbe.15-02--0043. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Guthrie JT, Wigfield A, VonSecker C. Effects of integrated instruction on motivation and strategy use in reading. Journal of Educational Psychology. 2000;92:331–341. doi: 10.1037/0022-0663.92.2.331. [DOI] [Google Scholar]
- Harackiewicz JM, Canning EA, Tibbetts Y, Priniski SJ, Hyde JS. Closing achievement gaps with a utility-value intervention: Disentangling race and social class. Journal of Personality and Social Psychology. 2016;111:745–765. doi: 10.1037/pspp0000075. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Harackiewicz JM, Priniski SJ. Improving student outcomes in higher education: The science of targeted intervention. Annual Review of Psychology. 2018;69:11.1–11.27. doi: 10.1146/annurev-psych-122216-011725. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Hernandez PR, Schultz PW, Estrada M, Woodcock A, Chance RC. Sustaining optimal motivation: A longitudinal analysis of interventions to broaden participation of underrepresented students in STEM. Journal of Educational Psychology. 2013;105:89–107. doi: 10.1037/a0029691. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Hu LT, Bentler PM. Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling: A Multidisciplinary Journal. 1999;6:1–55. doi: 10.1080/10705519909540118. [DOI] [Google Scholar]
- Hulleman CS, Barron KE. Motivation interventions in education: Bridging theory, research, and practice. In: Corno L, Anderman E, editors. Handbook of Educational Psychology. 3. New York, NY: Taylor & Francis; 2016. pp. 160–171. [Google Scholar]
- Hulleman CS, Harackiewicz JM. Promoting interest and performance in high school science classes. Science. 2009;326:1410–1412. doi: 10.1126/science.1177067. [DOI] [PubMed] [Google Scholar]
- Hulleman CS, Kosovich JJ, Barron KE, Daniel DB. Making connections: Replicating and extending the utility value intervention in the classroom. Journal of Educational Psychology. 2017;109:387–404. doi: 10.1037/edu0000146. [DOI] [Google Scholar]
- Jones MT, Barlow AEL, Villarejo M. Importance of undergraduate research for minority persistence and achievement in biology. The Journal of Higher Education. 2010;81:82–115. doi: 10.1353/jhe.0.0082. [DOI] [Google Scholar]
- Kaplan A, Garner JK. A complex dynamic systems perspective on identity and its development: The dynamic systems model of role identity. Developmental Psychology. 2017;53:2036–2051. doi: 10.1037/dev0000339. [DOI] [PubMed] [Google Scholar]
- Kline RB. Principles and practice of structural equation modeling. New York: Guilford Press; 2011. [Google Scholar]
- Larson LM, Pesch KM, Surapaneni S, Bonitz VS, Wu TF, Werbel JD. Predicting graduation: The role of mathematics/science self-efficacy. Journal of Career Assessment. 2015;23:399–409. doi: 10.1177/1069072714547322. [DOI] [Google Scholar]
- Lazowski RA, Hulleman CS. Motivation interventions in education: A meta-analytic review. Review of Educational Research. 2016;86:602–640. doi: 10.3102/0034654315617832. [DOI] [Google Scholar]
- Linnenbrink-Garcia L, Patall EA, Messersmith EE. Antecedents and consequences of situational interest. British Journal of Educational Psychology. 2013;83:591– 614. doi: 10.1111/j.2044-8279.2012.02080.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Linnenbrink-Garcia L, Patall EA, Pekrun R. Adaptive motivation and emotion in education: Research and principles for instructional design. Policy Insights from the Behavioral and Brain Sciences. 2016;3:228–236. doi: 10.1177/2372732216644450. [DOI] [Google Scholar]
- Linnenbrink-Garcia L, Wormington SV. Key challenges and potential solutions for studying the complexity of motivation in schooling: An integrative, dynamic person-oriented perspective. British Journal of Educational Psychology Monograph Series. 2017;89:108. [Google Scholar]
- Maltese AV, Tai RH. Pipeline persistence: Examining the association of educational experiences with earned degrees in STEM among US students. Science Education. 2011;95:877–907. doi: 10.1002/sce.20441. [DOI] [Google Scholar]
- Maton KI, Domingo MRS, Stolle-McAllister KE, Zimmerman JL, Hrabowski FA., III Enhancing the number of African-Americans who pursue STEM PhDs: Meyerhoff Scholarship Program outcomes, processes, and individual predictors. Journal of Women and Minorities in Science and Engineering. 2009;15:15–37. doi: 10.1615/JWomenMinorScienEng.v15.i1.20. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Maxwell SE, Cole DA, Mitchell MA. Bias in cross-sectional analyses of longitudinal mediation: Partial and complete mediation under an autoregressive model. Multivariate Behavioral Research. 2011;46:816–841. doi: 10.1080/00273171.2011.606716. [DOI] [PubMed] [Google Scholar]
- McGee R, Keller JL. Identifying future scientists: Predicting persistence into research training. CBE – Life Sciences Education. 2007;6:316–331. doi: 10.1187/cbe.07-04-0020. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Meece JL, Glienke BB, Burg S. Gender and motivation. Journal of School Psychology. 2006;44:351–373. doi: 10.1016/j.jsp.2006.04.004. [DOI] [Google Scholar]
- Muthén LK, Muthén BO. Mplus. The comprehensive modeling program for applied researchers: User’s guide. 2012;7 [Google Scholar]
- National Science Foundation, National Center for Science and Engineering Statistics. Women, minorities, and persons with disabilities in science and engineering: 2015. 2015 (Special Report NSF 15–311). Retrieved from http://www.nsf.gov/statistics/wmpd/
- Palmer DH. Student interest generated during an inquiry skills lesson. Journal of Research in Science Teaching. 2009;46:147–165. doi: 10.1002/tea.20263. [DOI] [Google Scholar]
- Patall EA, Cooper H, Wynn SR. The effectiveness and relative importance of choice in the classroom. Journal of Educational Psychology. 2010;102:896–915. doi: 10.1037/a0019545. [DOI] [Google Scholar]
- Paunesku D, Walton GM, Romero C, Smith EN, Yeager DS, Dweck CS. Mind-set interventions are a scalable treatment for academic underachievement. Psychological Science. 2015;26:784–793. doi: 10.1177/0956797615571017. [DOI] [PubMed] [Google Scholar]
- Perez T, Cromley JG, Kaplan A. The role of identity development, values, and costs in college STEM retention. Journal of Educational Psychology. 2014;106:315–329. doi: 10.1037/a0034027. [DOI] [Google Scholar]
- Pintrich PR. A motivational science perspective on the role of student motivation in learning and teaching contexts. Journal of Educational Psychology. 2003;95:667–686. doi: 10.1037/0022-0663.95.4.667. [DOI] [Google Scholar]
- President’s Council of Advisors on Science and Technology. Engage to excel: Producing one million additional college graduates with degrees in science, technology, engineering, and mathematics. 2012 Feb; (Report to the President). Retrieved from http://www.whitehouse.gov/sites/default/files/microsites/ostp/pcast-engage-to-excel-final_feb.pdf.
- Pugh K, Linnenbrink-Garcia L, Philips M, Perez AC. Supporting the development of transformative experience and interest. Chapter to appear. In: Renninger KA, Nieswandt M, Hidi S, editors. Interest, the self, and K-16 mathematics and science learning. Washington, DC: AERA; 2015. pp. 369–384. [Google Scholar]
- Raelin JA, Bailey MB, Hamann J, Pendleton LK, Reisberg R, Whitman DL. The gendered effect of cooperative education, contextual support, and self-efficacy on undergraduate retention. Journal of Engineering Education. 2014;103:599–624. doi:002/jee.20060. [Google Scholar]
- Raykov T. Evaluation of scale reliability for unidimensional measures using latent variable modeling. Measurement and Evaluation in Counseling and Development. 2009;42:223–232. doi: 10.1177/0748175609344096. [DOI] [Google Scholar]
- Rayman P, Brett B. Women science majors: What makes a difference in persistence after graduation? The Journal of Higher Education. 1995;66:388–414. doi: 10.2307/2943794. [DOI] [Google Scholar]
- Renninger KA, Hidi S. Student interest and achievement: Developmental issues raised by a case study. In: Wigfield A, Eccles JS, editors. Development of achievement motivation. New York: Academic Press; 2002. pp. 173–195. [Google Scholar]
- Reeve J, Deci EL, Ryan RM. Self-determination theory: A dialectical framework for understanding sociocultural influences on student motivation. In: McInerney DM, Van Etten S, editors. Big theories revisited. Greenwich, CT: Information Age Press; 2004. pp. 31–60. [Google Scholar]
- Renninger KA, Hidi S. Student interest and achievement: Developmental issues raised by a case study. In: Wigfield A, Eccles JS, editors. Development of achievement motivation. San Diego, CA: Academic Press; 2002. pp. 173–195. [Google Scholar]
- Rosenbaum PR, Rubin DB. The central role of the propensity score in observational studies for causal effects. Biometrika. 1983;70:41–55. doi: 10.1093/biomet/70.1.41. [DOI] [Google Scholar]
- Rosenzweig EQ, Wigfield A. STEM motivation interventions for adolescents: A promising start, but further to go. Educational Psychologist. 2016;51:146–163. doi: 10.1080/00461520.2016.1154792. [DOI] [Google Scholar]
- Ryan RM, Deci EL. Self-determination theory and the facilitation of intrinsic motivation, social development, and well-being. American Psychologist. 2000;55:68–78. doi: 10.1037/0003-066X.55.1.68. [DOI] [PubMed] [Google Scholar]
- Russell SH, Hancock MP, McCullough J. Benefits of undergraduate research experiences. Science. 2007;316:548–549. doi: 10.1126/science.1140384. [DOI] [PubMed] [Google Scholar]
- Schiefele U. The role of interest in motivation and learning. In: MCollis JM, Messick S, editors. Intelligence and personality: Bridging the gap in theory and measurement. Mahwah, NJ: Lawrence Erlbaum Associates; 2001. pp. 163–194. [Google Scholar]
- Schunk DH, Pajares F. Competence beliefs in academic functioning. In: Elliot AJ, Dweck C, editors. Handbook of Competence and Motivation. New York: Guilford Press; 2005. pp. 85–104. [Google Scholar]
- Schultz PW, Hernandez PR, Woodcock A, Estrada M, Chance RC, Aguilar M, Serpe RT. Patching the pipeline reducing educational disparities in the sciences through minority training programs. Educational Evaluation and Policy Analysis. 2011;33:95–114. doi: 10.3102/0162373710392371. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Seymour E. The loss of women from science, mathematics, and engineering undergraduate majors: An explanatory account. Science Education. 1995;79:437–473. doi: 10.1002/sce.3730790406. [DOI] [Google Scholar]
- Seymour E, Hewitt NM. Talking about leaving: Why undergraduates leave the sciences. Boulder, Colo: Westview Press; 1997. [Google Scholar]
- Siegle D, McCoach DB. Increasing student mathematics self-efficacy through teacher training. Journal of Advanced Academics. 2007;18:278–312. doi: 10.4219/jaa-2007-353. [DOI] [Google Scholar]
- Skinner EA, Kindermann TA, Connell JP, Wellborn JG. Engagement and disaffection as organizational constructs in the dynamics of motivational development. In: Wentzel K, Wigfield A, editors. Handbook of motivation in school. Mahwah, NJ: Lawrence Erlbaum; 2009. pp. 223–245. [Google Scholar]
- Stolle-McAllister K, Domingo MRS, Carrillo A. The Meyerhoff way: How the Meyerhoff scholarship program helps black students succeed in the sciences. Journal of Science Education and Technology. 2011;20:5–16. doi: 10.1007/s10956-010-9228-5. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Stuart EA. Matching methods for causal inference: A review and a look forward. Statistical Science. 2010;25:1–21. doi: 10.1214/09-STS313. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Thompson MS, Green SB. Evaluating between-group differences in latent variable means. In: Hancock GR, Mueller RO, editors. Structural equation modeling: A second course. Charlotte, NC: Information Age Publishing Inc; 2013. pp. 163–218. [Google Scholar]
- Trautwein U, Nagengast B, Marsh HW, Gaspard H, Dicke A-L, Lüdtke O, Jonkmann K. Expectancy-Value Theory revisited: From Expectancy-Value Theory to Expectancy-ValueS Theory? In: McInerney DM, Marsh HW, Craven RG, Guay F, editors. International advances in self research. Theory driving research: New wave perspectives on self-processes and human development. Charlotte, NC: IAP Information Age Publishing; 2013. pp. 233–249. [Google Scholar]
- Turner JC, Patrick H. How does motivation develop and why does it change? Reframing motivation research. Educational Psychologist. 2008;43:119–131. doi: 10.1080/00461520802178441. [DOI] [Google Scholar]
- Turner JC, Warzon KB, Christensen A. Motivating mathematics learning changes in teachers’ practices and beliefs during a nine-month collaboration. American Educational Research Journal. 2011;48:718–762. doi: 10.3102/0002831210385103. [DOI] [Google Scholar]
- Usher EL. Sources of middle school students’ self-efficacy in mathematics: A qualitative investigation. American Educational Research Journal. 2009;46:275–314. doi: 10.3102/0002831208324517. [DOI] [Google Scholar]
- Villarejo M, Barlow AEL, Kogan D, Veazey BD, Sweeney JK. Encouraging minority undergraduates to choose science careers: Career Paths Survey results. CBE-Life Sciences Education. 2008;7:394–409. doi: 10.1187/cbe.08-04-0018. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Walton GM. The new science of wise psychological interventions. Current Directions in Psychological Science. 2014;23:73–82. doi: 10.1177/0963721413512856. [DOI] [Google Scholar]
- Wang MT, Degol J. Motivational pathways to STEM career choices: Using expectancy-value perspective to understand individual and gender differences in STEM fields. Developmental Review. 2013;33:304–340. doi: 10.1016/j.dr.2013.08.001. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Wigfield A, Cambria J. Expectancy-value theory: Retrospective and prospective. In: Urdan TC, Karabenick SA, editors. The decade ahead: Theoretical perspectives on motivation and achievement. Advances in motivation and achievement. 16A. Bingley, UK: Emerald Group Publishing Limited; 2010. pp. 35–70. [Google Scholar]
- Wigfield A, Eccles J. The development of competence beliefs, expectancies for success, and achievement values from childhood through adolescence. In: Wigfield A, Eccles J, editors. Development of Achievement Motivation. San Diego, CA: Academic Press; 2002. pp. 91–120. [Google Scholar]
- Yeager DS, Romero C, Paunesku D, Hulleman CS, Schneider B, Hinojosa C, Dweck CS. Using design thinking to improve psychological interventions: The case of the growth mindset during the transition to high school. Journal of Educational Psychology. 2016;108:374–391. doi: 10.1037/edu0000098. [DOI] [PMC free article] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.