Abstract
To increase the numbers of underrepresented racial minority students in science, technology, engineering, and mathematics (STEM), federal and private agencies have allocated significant funding to undergraduate research programs, which have been shown to students’ intentions of enrolling in graduate or professional school. Analyzing a longitudinal sample of 4,152 aspiring STEM majors who completed the 2004 Freshman Survey and 2008 College Senior Survey, this study utilizes multinomial hierarchical generalized linear modeling (HGLM) and propensity score matching techniques to examine how participation in undergraduate research affects STEM students’ intentions to enroll in STEM and non-STEM graduate and professional programs. Findings indicate that participation in an undergraduate research program significantly improved students’ probability of indicating plans to enroll in a STEM graduate program.
Introduction
Freshman college students’ initial interest in majoring in science, technology, engineering, and mathematics (STEM) disciplines has increased in recent years, and underrepresented racial minority (URM) students appear just as interested in these fields as their White and Asian American counterparts (Higher Education Research Institute [HERI], 2010). Even with undergraduates’ renewed interest in majoring in STEM, bachelor’s degree completion rates in these areas remain persistently low, especially among URM students (Center for Institutional Data Exchange and Analysis, 2000; HERI, 2010). The lost STEM talent among URM students becomes even more pronounced when considering graduate enrollment in STEM, as American Indian, Black, and Latino students represented just 0.4%, 4.9%, and 3.6%, respectively, of all STEM graduate students during the 2006-2007 academic year (Council of Graduate Schools, 2007).
To increase the representation of all students, and particularly American Indian, Black, and Latino students, in STEM graduate programs, federal agencies, such as the National Institutes of Health (NIH) and the National Science Foundation (NSF), have invested significantly in undergraduate research programs geared toward retaining students in undergraduate STEM disciplines and facilitating their aspirations for and matriculation into STEM graduate programs. These investments in undergraduate research programs serve not only to diversify the pool of scientific researchers but also to maintain if not increase the nation’s scientific capacity for research and innovation. Prior studies examining the benefits of undergraduate research programs have concluded that these programs represent an important catalyst for increasing students’ commitment to pursuing STEM graduate programs (e.g., Hunter, Laursen, & Seymour, 2007; Laursen, Seymour, Hunter, Thiry, & Melton, 2010; Lopatto, 2004; MacLachlan, 2006; Russell, Hancock, & McCullough, 2007; Seymour, Hunter, Laursen, & DeAntoni, 2004); however, many of these studies have serious shortcomings, which range from limited generalizability due to data collected from single-institution samples to over-estimation of the effect of undergraduate research programs by relying on simple descriptive statistics that fail to account for potential endogeneity in the data.
By relying on descriptive statistics, prior studies may have misestimated the short- and long-term benefits of participation in an undergraduate research program, particularly as they relate to students’ educational aspirations and graduate enrollment outcomes. Drawing from a national sample of initial STEM aspirants in four-year colleges and universities, this study uses multivariate analyses to estimate the relationship between participation in an undergraduate research program and students’ plans to enroll in either a STEM graduate program or a non-STEM graduate program relative to students who have no intentions for post-baccalaureate study. Given that the federal government, private agencies (e.g. Howard Hughes Medical Institute), and individual institutions have invested substantial funding in undergraduate research programs with a goal of improving the educational success of STEM students, this study examines how participation in these programs relates to students’ graduate and professional school enrollment intentions through the use of advanced statistical techniques.
Why Study Aspirations: The Relation of Aspirations to Enrollment
College choice literature suggests that developing a predisposition for advanced education represents the first phase of matriculating into a program (Hossler & Gallagher, 1987; Perna, 2006). Individuals’ predispositions lead to search and eventually college choice. Other research has found that a student’s educational aspiration represents one of the strongest predictors of subsequent enrollment in an undergraduate or graduate degree program (Heller, 2001; Mullen, Goyette, & Soares, 2003; Nevill & Chen, 2007; Sewell, Haller, & Portes, 1969; Walpole, 2003). In examining predictors of graduate enrollment, Heller (2001) notes that, “The most influential factor was a student’s degree expectations” (p. 29). Heller (2001) found that individuals with bachelor’s degree aspirations were 16 percentage points less likely to enroll in graduate school compared to their peers with intentions for a master’s degree or MBA. By contrast, Heller found that students with plans for a first professional were 30 points more likely to enroll in graduate school compared to their peers with master’s degree aspirations. Likewise, those with intentions for a doctoral degree were 28 points more likely than their counterparts with master’s degree aspirations to enroll in graduate school.
Walpole (2003) analyzed national data from the Higher Education Research Institute (HERI) to identify predictors of enrolling in graduate or professional school within nine years of initial matriculation into an undergraduate program. She found that college seniors with plans to pursue graduate school were 14.35 times as likely to enroll in a graduate or professional program compared to their peers without these aspirations. Across all variables in the model, which included college experiences (e.g., time spent studying, faculty interaction, GPA) and pre-college characteristics (e.g., SAT scores, race, gender, pre-college aspirations), students’ senior year intentions to pursue a graduate degree represented the strongest predictor of eventual enrollment in a graduate or professional program.
Findings from Nevill and Chen’s (2007) analysis of Baccalaureate and Beyond (92/03) data suggest that 16.7% of bachelor’s degree recipients without graduate enrollment intentions enrolled in a graduate or professional program within 10 years of completing their bachelor’s degrees, yet the authors do not break out these findings by discipline. If students do not have aspirations to pursue STEM graduate or professional degrees by the end of college, it is not likely they will pursue graduate school in STEM. They likely will have missed opportunities for using faculty networks to gain access to graduate school. By contrast, Nevill and Chen (2007) report that roughly half of degree earners who expressed graduate enrollment plans in 1993 enrolled in a graduate or professional program by 2003, but these false positives may continue to decline over the cohort’s lifespan. Nevill and Chen (2007) conclude that degree earners with aspirations for graduate degrees have significantly greater odds of enrolling in a graduate or professional program than their peers who do not have such aspirations.
These findings connecting aspirations to future enrollment relate to college choice theory (e.g., Hossler & Gallagher, 1987; Perna, 2006), which suggests that students’ development of predispositions to pursue advanced education represents the necessary first step in deciding to enroll. College choice scholars emphasize that interest drives future action, as individuals predisposed to pursuing advanced education have a much greater likelihood of actually enrolling in advanced degree programs. Given this research, the present study examines how participating in an undergraduate research program affects students’ intentions to enroll in a graduate or professional program.
Status Attainment Theory and Pre-College Characteristics
Status attainment theory provides a useful lens toward understanding the factors that influence individuals’ intentions to pursue advanced levels of education. Status attainment theory posits that educational aspirations are a function of individuals’ background characteristics, sense of origin, prior academic achievement, and the influence of significant others (Blau & Duncan, 1967; Sewell, Haller, & Portes, 1969).
Background characteristics include measures of race and gender whereas sense of origin typically refers to an individual’s social standing, as measured by parents’ education and income. These characteristics contribute to students’ likelihood of enrolling or planning to enroll in graduate or professional programs (Blau & Duncan, 1967; Carter, 1999, 2001; Nevill & Chen, 2007). For example, research suggests that students’ background characteristics, including gender, race, parental education, and socioeconomic status, significantly predict graduate school aspirations (Carter, 2001), decisions to apply to graduate school (Perna, 2004), and enrollment in graduate programs (Nevill & Chen, 2007; Perna, 2004). With regard to gender, larger proportions of women than men enroll in programs at the master’s degree level but smaller proportions of women enroll in doctoral and first professional degree programs (Perna, 2004). Looking specifically at STEM students, Sax (2001) found that just 25% of women who earned bachelor’s degrees in science, math, or engineering matriculated into graduate programs in these disciplines compared to 32% of men.
Findings connecting race with graduate degree aspirations or graduate enrollment appear to be more mixed (Heller, 2001; Millett, 2003). Millet (2003) found that Black students were significantly less likely than their White peers to enroll in their first-choice graduate or professional school, but she found no difference between White students and their Latino or Asian American peers. By contrast, Heller (2001) found that Black students were significantly more likely than their White counterparts to enroll in graduate school within four years of completing their bachelor’s degree; however, this relationship became non-significant after accounting for institutional type, college experiences, and overall debt.
The effects of race on graduate aspirations and enrollment may be confounded by college generational status and socioeconomic status, which is often operationalized as a combination of parental education and financial resources. Status attainment theory refers to socioeconomic status as an individual’s sense of origin. Students from families with higher income and parents with greater levels of education report having more resources at their disposal and significantly higher degree aspirations than their peers from lower socioeconomic backgrounds. The role of socioeconomic status in influencing students’ degree aspirations appears especially salient for students of color (Burke & Hoelter, 1988; Carter, 1999, 2001), as socioeconomic status brings with it access to other forms of capital, including social and cultural. Additionally, students whose parents earned a college degree or higher typically have greater likelihoods of pursuing post-baccalaureate degrees (Heller, 2001; Mullen et al., 2003).
Status attainment theory also suggests that prior academic achievement predicts intentions to pursue advanced levels of education. Measure of academic achievement, such as measured by SAT scores, GRE scores, high school GPAs, and undergraduate GPAs, have been identified as important predictors of graduate school enrollment (Heller, 2001; Millett, 2003; Mullen, Goyette, & Soares, 2003; Nevill & Chen, 2007) and post-baccalaureate degree aspirations (Buchmann & Dalton, 2002; Carter, 2001; Pascarella et al., 2004). For example, Pascarella et al. (2004) concluded that tested academic ability prior to college had a significant positive relationship with degree aspirations for White students but had no significant predictive power for Latino or Black students.
College Student Socialization and College Context
Although status attainment theory provides an important perspective in understanding students’ development of aspirations for post-baccalaureate education, this lens does not consider the influence of college on students. In addition to being influenced by background characteristics and pre-college achievement, students encounter a socialization process in college that affects their intentions to pursue post-baccalaureate degrees (Weidman, 1989). Weidman’s (1989) theory of college student socialization posits that, as students become socialized into a particular environment, they may begin to be affected by the norms of that social structure. Under Weidman’s model, students who socialize with and compare themselves to a set of high-achieving students feel more pressure to mirror the higher levels of achievement of their peers. One context where peers may compare themselves with other high-achieving students is within undergraduate research programs where students are often selected for participation through a competitive process.
Weidman (1989) offers a similar suggestion for relationships and interactions between students and faculty. Students who interact with faculty more frequently report aspirations for postsecondary degrees (Carter, 2001; Hearn, 1987). Researchers have found this relationship particularly salient for URM students, as more frequent interactions with faculty among URM students corresponds with significantly higher degree aspirations (Carter, 2002; Maton, Hrabowski, & Schmitt, 2000). For undergraduates in STEM, these interactions are often enhanced through undergraduate research opportunities.
In addition to having increased contact with faculty, high-achieving students have significantly greater likelihoods of reporting aspirations to enroll in graduate or professional programs (Carter, 1999, 2001; Pascarella, 1984; Sax, 2001). Weidman (1989) emphasizes academic aptitude prior to college as well as students’ formal academic socialization within their major departments as important in predicting their development of future aspirations. This relationship is expected given that graduate programs base admission decisions upon students’ academic record and their likelihood of success as a graduate student (Griffin & Muniz, 2011).
Although individual agency, as measured by ability, interactions with significant others, and ambition, significantly influences the extent to which students develop aspirations for post-baccalaureate study, the institutional context also shapes individuals’ degree plans. Weidman’s (1989) model emphasizes the role of normative peer pressures within the institution that help to shape students’ goals. More selective institutions may also provide an environment in which students become more socialized toward the pursuit of graduate or professional degrees (Mullen, Goyette, & Soares, 2003). Additionally, historically Black colleges and universities (HBCUs) have been cited for their production of Black students going into graduate programs (Stage, John, & Hubbard, 2011), particularly because of the mentorship and institutional ethos available on these campuses (Allen, 1992). These contextual influences, however, have not been examined in connection with undergraduate research programs and the role these programs have in promoting students’ plans for graduate school.
Science Identity and Undergraduate Research Programs
For many STEM students, their socialization in college toward developing graduate degree aspirations may come from participating in undergraduate research programs. Undergraduate research programs socialize students by connecting them with faculty and advanced peers who provide undergraduates with access to professional networks and new sources of information, and broader access to institutional resources and networks improves students’ capacity to navigate the educational system (Lin, 2001; McDonough, 1997, 1998). These programs typically pair students with a faculty mentor who not only provides direction in the research process but also serves as an institutional agent who can assist the student in establishing key connections across the institution and the discipline (Laursen et al., 2010). According to Hurtado, Cabrera, Lin, Arellano, and Espinosa (2009), undergraduate research initiatives with support systems that provide high levels of mentoring and peer relationships and that acquaint students with scientific norms better enable students to access opportunities at their undergraduate institutions that will develop their science orientation. In many cases, these programs also provide activities that orient students toward preparing for graduate school.
Undergraduate research programs also develop a stronger identification with participants’ respective STEM disciplines, which can help orient them toward graduate and professional programs in science and engineering. According to Carlone and Johnson (2007), students’ science identities can be strengthened in three key ways: (1) by fostering knowledge growth, (2) by providing opportunities to socially display scientific knowledge and practices, and (3) by building one’s acknowledgement as being a “science person,” especially by way of recognition by others (including faculty who have their own scientific networks). Research programs may improve students’ likelihood of developing post-baccalaureate degree aspirations and eventual enrollment in graduate studies by providing them with the opportunity to perform as scientists by conducting original research rather than “cookbook” labs where the outcome is predetermined (Seymour et al., 2004). Conducting authentic research may generate interest among students in discovering new knowledge and the career opportunities associated with knowledge discovery. Additionally, undergraduate research opportunities provide students with the space to begin envisioning themselves as scientists, as they consider what a career in STEM research may resemble by observing their faculty mentors. Finally, research programs further develop students’ science skills, allowing them to develop the competencies necessary to be successful in STEM (Seymour et al., 2004).
Participation in undergraduate research provides a unique opportunity for URM students in developing their science identities. Science identity development, as originally conceived by Carlone & Johnson (2007), highlights the process of being recognized as a legitimate scientist among those who are at the margins of science disciplines (i.e. women of color). Students’ science identity development can be conflicted by negative stereotypes or lack of recognition from the scientific community (Tran, 2011). Additionally, Tran (2011) notes how students must negotiate among multiple identities, including their identity as a scientist. Therefore, science identity specifically relates to our examination of undergraduate research as a mechanism for promoting structured faculty mentorship for URM students in STEM. These structured programs may also provide students with critical support and information that helps students navigate STEM pathways and serve as formal spaces for demonstrating science discourse and practice, along with other opportunities for gaining recognition as a scientist (Carlone & Johnson, 2007). These programs may also provide the space for students to learn how their work as a scientist can have real impacts on local communities with which they identify.
Limitations of Previous Studies on Undergraduate Research Participation
Although previous studies have documented the benefits of undergraduate research programs, the vast majority of scholarship on the student-derived benefits from undergraduate research participation has analyzed data collected from single institutions and individual programs, and such approaches limit the generalizability of the findings to other institutions and initiatives. These studies also rely on interpretive rather than correlational or multivariate analyses. Furthermore, previous studies have tended to be retrospective in nature by asking alumni from undergraduate research programs to discuss their experiences or to identify the key undergraduate opportunities that enabled them to pursue graduate school (e.g., Barlow & Villarejo, 2004; Bauer & Bennett, 2003; Hathaway, Nagda, & Gregerman, 2002; MacLachlan, 2006). Other studies have used simple comparisons between undergraduate research program participants and nonparticipants in examining graduate school enrollment rates (e.g., Lopatto, 2004; Maton & Hrabowski, 2004). A third category of studies have used qualitative and quantitative longitudinal research designs to examine the influences of undergraduate research experiences over time (Hunter et al., 2007; Laursen et al., 2010; Russell et al., 2007; Seymour et al., 2004), yet none of these studies has accounted for the potential selection bias inherent in the data, as students do not randomly decide to participate in undergraduate research programs.
Bauer and Bennett (2003) conducted a retrospective cross-sectional study of alumni from a mid-sized, research-intensive institution and found that 80% of the alumni who had participated in an undergraduate research program also had enrolled in graduate school, and this figure compared to just 59% of the alumni who did not participate in an undergraduate research program. Similarly, Hathaway et al. (2002) compared three groups of alumni: those who had participated in (1) a university-sponsored Undergraduate Research Opportunity Program (UROP), (2) an alternative research experience on campus, or (3) no research activity during their undergraduate tenure. The authors reported that 81.5% of UROP alumni and 82% of alumni who had an alternative research experience pursued graduate education, whereas only 65.4% of alumni who did not engage in research went on to graduate school.
Using simple comparisons of the graduate school enrollment rates between program participants and non-participants in analyses of cross-sectional data likely overstates the effect of undergraduate research programs on students’ subsequent graduate school enrollment. This over-estimation primarily occurs because undergraduate research programs typically have GPA cut-offs and rely on faculty to identify students who are often interviewed prior to program participation; thus, these programs typically recruit higher-achieving students who likely have a greater propensity to pursue graduate or professional school. Furthermore, single-institution studies offer policymakers limited information about the effectiveness of undergraduate research programs from a broader national perspective. Although these retrospective studies provide important insight into individuals’ perceived benefits from having participated in an undergraduate research program, these studies lack specific controls or covariates that might account for additional motivational factors that relate to students’ future aspirations, success, and inclination to participate in an undergraduate research program.
Other researchers have relied on analyses of longitudinal qualitative or quantitative data to examine the effectiveness of undergraduate research programs. Russell et al. (2007) surveyed research undergraduates participating in a research program in 2003 and 2005, and the authors concluded that, after completing the program, 29% of research program participants reported new ambitions for earning a Ph.D. Additionally, 68% of participants indicated having a stronger interest in a STEM career. Although Russell et al. (2007) used a longitudinal design in their research, their study utilized simple descriptive statistics and lacked a comparison group (i.e., students who did not participate in undergraduate research programs). Without a comparison group, we cannot attribute gains in STEM career interest or commitment to earning a Ph.D. to students’ participation in research.
In their longitudinal qualitative study, Seymour et al. (2004) documented why participating in an undergraduate research program might improve students’ interest in pursuing graduate studies. They found that students tend to connect their experiences in research programs with increased confidence in conducting research, defending their findings, and making contributions to their discipline. Research program participants also reported having gained a deeper level of knowledge and understanding of scientific theory and concepts as well as an increase in critical thinking and problem-solving skills through their participation. Although Seymour et al. (2004) provide additional insight into the reasons undergraduate research programs make a difference for students, alternative explanations for students’ success have not effectively been ruled out, and we have limited evidence as to the measurable effect of program participation on students’ commitment to post-baccalaureate studies.
This body of literature has demonstrated that STEM students who participate in undergraduate research have a higher likelihood of aspiring to graduate education or enrolling in graduate programs, yet many of these studies generally have focused on one institution or a single program. Additionally, past studies have been limited by their use of simple descriptive comparisons between research participants and non-participants, and such an approach ignores possible differences between program participants and non-participants. Drawing from advanced statistical techniques and a national longitudinal dataset, this study addresses the methodological shortcomings of previous studies to examine the effectiveness of undergraduate research programs on influencing students’ intentions to pursue a graduate or professional program in STEM disciplines.
Methodology
Sample
We examined the relationship between undergraduate research participation and graduate school enrollment intentions by analyzing a longitudinal sample that comes from the 2004 Freshman Survey (TFS) and 2008 College Senior Survey (CSS), both of which were administered by the Cooperative Institutional Research Program (CIRP) at HERI. CIRP’s TFS asked entering freshmen about their goals, high school experiences, values, and perspectives on an array of political and social issues. CIRP’s CSS collected data about students’ college experiences, future plans, and post-tests on students’ goals and values.
Funding from NIH and NSF allowed for a sampling process that targeted specific institutions and students within those institutions to participate in the longitudinal study. We targeted colleges and universities with strong reputations for producing high numbers of STEM bachelor’s degrees, institutions that had undergraduate research programs funded by NSF and NIH, and a number of minority-serving institutions (MSIs) that do not normally participate in CIRP-sponsored surveys. The longitudinal sample for this study includes 4,152 students from all racial and ethnic backgrounds who indicated in 2004 an intention to pursue a STEM-related bachelor’s degree. The institutional sample includes 219 four-year colleges and universities.
Variables
The dependent variable is a three-part categorical variable regarding students’ intentions to pursue a STEM-related graduate or professional program, a non-STEM related graduate or professional program, or a path that does not include graduate or professional school. We derived this variable from a question on the CSS that asked students to report their intended graduate school major1, and students who did not plan to enroll in graduate school were instructed to skip the question. Appendix A has a full list of the dependent and independent variables for the study.
Previous studies have relied upon a single measure of students’ degree aspirations or intentions (e.g., Carter, 1999, 2001; Pascarella et al., 2004). Others have demonstrated that students’ self-reported degree aspirations represent one of the strongest predictors of actual enrollment in graduate or professional programs (e.g., Heller, 2001; Mullen, Goyette, & Soares, 2003; Nevill & Chen, 2007; Walpole, 2003). Students who lack the ambition or desire to pursue post-baccalaureate degrees have substantially reduced odds of enrolling in graduate or professional programs2; thus, examining the relationship between research participation and intentions to pursue graduate study represents an important first step in beginning to understand how research participation may relate to eventual enrollment in post-baccalaureate degree programs.
The primary independent variable of interest is whether students participated in a structured undergraduate research program during college, which is taken from a dichotomous item on the 2008 CSS. The remaining predictors are included as possible alternative explanations as to why students report intentions to pursue graduate study in STEM and non-STEM fields, and these measures align with the frameworks guiding this study. The full model controls for student background characteristics, such as race, gender, income, and mother’s education, as status attainment theory suggests that these characteristics significantly affect individuals’ development of advanced educational aspirations (Carter, 1999, 2001; Sewell, Haller, & Portes, 1969). Likewise, status attainment theory emphasizes the role of pre-college preparation in predicting individuals’ educational goals, so we include measures of students’ high school GPA, the number of years they studied math in high school, and composite SAT scores.
Status attainment theory (Sewell, Haller, & Portes, 1969) and Weidman’s (1989) model of college student socialization underscore the influence of individuals’ dispositions on the development of future career and educational goals. Additionally, Carter (1999, 2001) and others have found that initial degree expectations represent strong predictors of later degree aspirations; therefore, the model accounts for students’ freshman-year aspirations for academic and medical doctorates compared to aspirations for other types of degrees. Similarly, we examined the relationship between wanting to come to college to prepare for graduate or professional study and students’ graduate school intentions in 2008.
Weidman’s (1989) model of college student socialization emphasizes the role of peers and faculty in shaping students’ goals and values during college. We examined how students’ interactions with graduate students and teaching assistants relate to their graduate and professional school intentions as seniors. Additionally, the full model includes measures of students’ self-reported hours per week spent studying and whether they joined a departmental academic club, as these activities demonstrate a commitment to academics and to students’ academic discipline. Receipt of faculty mentorship represents a construct scored by CIRP using item-response theory (Sharkness, DeAngelo, & Pryor, 2010) and represents a measure of the quality and types of students’ interactions with faculty. Academic major GPA is included as a representation of students’ academic achievement in college, as higher levels of achievement are expected to be associated with an increased likelihood in expressing intentions to pursue STEM or non-STEM graduate and professional programs (Carter, 1999, 2001; Pascarella, 1984). The model also includes measures of two career goals: working for social change and discovering new knowledge. Working for social change may be particularly salient for students who aspire toward non-STEM graduate or professional programs, and discovering new knowledge is likely to appeal to students intending to pursue post-baccalaureate degrees regardless of discipline given the original research associated with such programs. For students interested in STEM graduate programs, these variables may also be connected to their science identity (Tran, 2011).
The last set of predictors at the student level represents controls for students’ finances. We examined how students’ undergraduate debt predicts their intentions to pursue graduate and professional degrees, as prior research has found a negative relationship with undergraduate debt and the decision to pursue post-baccalaureate education (Heller, 2001; Malcom & Dowd, 2012). Additionally, we accounted for the amount of specific resources students used to finance their undergraduate education. These financial variables may serve as external pressures on students’ development of educational expectations (Weidman, 1989) and may affect whether they decide to invest further in their education (Winston, 1999).
Finally, the model includes several institutional predictors to examine college context. Weidman (1989) emphasizes the role of institutional context in shaping students’ goals and values. We included in the model measures of institutional control, selectivity, size, the proportion of White undergraduates, and the proportion of undergraduate STEM majors. Institutional control and selectivity are often associated with status, and prior research has identified a link between attending higher-status institutions and the decision to pursue graduate and professional education (Zhang, 2005). We also accounted for whether an institution was classified as a historically Black college or university (HBCU). Studies have indicated that HBCUs are more likely to be the undergraduate origin of Black doctoral recipients in science and engineering (Solorzano, 1995) and African American women’s successful post-baccalaureate achievements (Wolf-Wendel, 1998).
Analyses
In order to test the relationship between participation in an undergraduate research program and students’ graduate school intentions, we carefully considered the nonrandom nature of the data, specifically the likelihood that students self-selected to participate in such programs. Consequently, students who participate in undergraduate research programs may be qualitatively different from their peers who do not participate in such programs (Hurtado, Eagan, Cabrera, Lin, Park, & Lopez, 2008). To account for potential selection bias especially associated with the key independent variable, we utilized two analytic techniques: a multinomial HGLM after adjusting the sample using propensity score matching techniques and a single-model multinomial HGLM that included a set of pre-college, college entry, and college experience covariates. The latter multinomial HGLM analysis with covariates also enabled us to test for cross-level interaction effects. This overall approach is consistent with how other researchers have addressed selection bias (e.g., Shadish et al., 2008).
The issue of selection bias tends to arise in studies that rely upon analysis of ex post facto, or “after the fact,” data. Titus (2007) explains that endogeneity bias “occurs when predictors of an outcome are themselves associated with other unobserved or observed variables” (p. 489). Furthermore, Titus argued that sample selection bias may result from the lack of experimental designs, and failure to account for this bias may lead to inaccurate findings. To strengthen causal inferences, Desjardins, McCall, Ahlburg, and Moye (2002) emphasized the need for higher education research to account for issues of endogeneity when examining the effects of college, or specific programs, on an array of student outcomes.
Given that our study analyzes observational data, we relied upon the counterfactual framework advanced by Rosenbaum and Rubin (1983, 1984, 1985). Guo and Fraser (2010) describe a counterfactual as “a potential outcome, or the state of affairs that would have happened in the absence of the cause” (p. 24). With regard to this study, for a student who participated in a research program, the counterfactual is the hypothetical outcome (graduate enrollment intention) had that student not taken part in an undergraduate research program. By contrast, the counterfactual for a non-participant in research is the potential probability of reporting a graduate enrollment intention if that individual had been a part of a research program. Comparing students with more similar pretreatment characteristics allows the differences between research participants and non-participants in the outcome variable to be closer to what we would expect from a random assignment of students to each of the two groups (Schneider, Carnoy, Kilpatrick, Schmidt, & Shavelson, 2007). This use of a counterfactual framework requires the estimation of a propensity score and a reweighting of the data based on estimated propensity scores. We generated propensity scores for participation in an undergraduate research program from a set of covariates provided in Appendix C. After generating propensity scores, we conducted analyses to determine whether our sample was balanced, and the findings from these analyses are presented in Appendix C. After confirming the balancing property, we created treatment weights based on the estimated propensity score.
Because of limitations in analyzing a categorical outcome with the propensity score matching program (PSMATCH2) in Stata, we relied on reweighting techniques to statistically adjust the sample based on students’ likelihood of participation in an undergraduate research program (Hirano & Imbens, 2001; Nichols, 2007, 2008; Rosenbaum, 1987). In reweighting our sample, we relied on suggested calculations by Nichols (2008) and Guo and Fraser (2010) to create weights for the average treatment effect (ATE), the average treatment for the treated (ATT) effect, and the average treatment for the untreated (ATU) effect. ATE estimates the treatment effect for the entire sample whereas the ATT effect provides an estimate of the difference in an outcome between research participants and nonparticipants among individuals who had similar high probabilities of participating in a research program. By contrast, the ATU effect can be described as, among research program nonparticipants, the change in probability of reporting specific graduate school intentions if these individuals had participated in an undergraduate research program. Guo and Fraser (2010) and Nichols (2008) suggest the following calculation be used to generate a weight for the average treatment effect:
| (1) |
where W corresponds to the value of treatment (1,0) and ê(x) corresponds to the propensity score. The weight for the ATT effect is given by:
| (2) |
where W corresponds to the value of treatment (1,0) and ê(x) corresponds to the propensity score (Guo & Fraser, 2010). Finally, the weight for the ATU effect is calculated by:
| (3) |
where W corresponds to the value of treatment (1,0), ê(x) corresponds to the propensity score, and μ represents the proportion of students who participated in a research program (Nichols, 2008). We checked for and found a region of common support, and we did not detect any extreme weights between the treatment and control groups. The distribution of weights is available from the authors.
After creating these weights, we used multinomial hierarchical generalized linear modeling (HGLM) to examine how research program participation relates to students’ graduate school intentions. We ran the multinomial HGLMs with the three weights (ATE, ATU, and ATT) described above as well as unadjusted models to determine whether we would achieve significantly different results based upon the model specification. Shadish et al. (2008) suggest that propensity score models may not perform as well as more traditional models with observable characteristics as covariates. Additionally, we ran a final multinomial HGLM with a rich set of covariates that included college experiences to examine whether the relationship between undergraduate research participation and graduate school intentions remained after accounting for other facets of the college experience.
Multinomial HGLM is appropriate for analyses where the dependent variable has a non-ranked, categorical structure (Hosmer & Lemeshow, 2004), and using a hierarchical model accounts for the clustering effect of the data (Raudenbush & Bryk, 2002). Our outcome was a three-point categorical variable, and we had nested data, as students were clustered within institutions. We ran a total of five multinomial HGLMs: (1) using the ATE weight; (2) using the ATT weight; (3) using the ATU weight; (4) an unadjusted model with only the covariates used in the propensity score calculation (i.e., only pre-college and college entry variables); and (5) a model with a richer set of covariates that included college experiences, which were collected at the same time as the treatment variable. Using Petersen’s (1985) recommended calculation for continuous variables and Cruce’s (2009) formula for categorical variables, we report the results from significant parameter estimates as delta-p statistics, which can be interpreted as the change in probability of reporting an intention to pursue a graduate or professional program in STEM or non-STEM relative to having no post-baccalaureate intentions for every one-unit change in the independent variable.
Limitations
This study is limited in several ways. First, as with any study involving analysis of secondary data, we are limited by the survey items and their coding schemes. For example, the primary independent variable of interest, participation in an undergraduate research program, does not allow us to disentangle the quality or effects of different types of undergraduate research programs (e.g., NSF-funded, NIH-funded, institutionally based, etc.) nor how long students participated in such programs during college.
Additionally, we rely on students’ intentions to enroll in a specific graduate or professional program. Although this study uses intentions to enroll rather than actual enrollment in graduate or professional programs, research has consistently demonstrated that intentions to pursue graduate education represent the strongest predictors of actual enrollment (Heller, 2001; Mullen, Goyette, & Soares, 2003; Nevill & Chen, 2007; Sewell, Haller, & Portes, 1969; Walpole, 2003). By using graduate enrollment intentions rather than actual graduate school matriculation, it is possible to have false positives (those who express an intention for graduate study but never enroll) and false negatives (those who have no intention to pursue graduate study but later enroll). False positives are less of a concern, as more students with enrollment intentions matriculate into graduate school as time passes (Nevill & Chen, 2007). In addition to these concerns, the key independent variable (undergraduate research participation) and the outcome variable are measured at the same point in time, which means that these programs may develop students’ interests in pursuing graduate school or that students who seek out these programs already have decided to pursue graduate school and merely want to gain research experience to improve their chances of success.
Given the potential drawbacks of relying on a single survey item in our dependent variable, we conducted sensitivity analyses to examine how sensitive our findings, particularly related to undergraduate research participation, were to the operationalization of our dependent variable. Appendix B provides the results of these sensitivity analyses using the model with the full set of covariates. The operationalization of the dependent variable presented in the results section is as described in the variables section above, where we derived the outcome based on students’ responses to the question: “Please write in the code for your graduate major; please omit if you do not plan to attend graduate school.” Students who skipped this question were coded as not intending to pursue graduate or professional study, and students who provided a response were appropriately placed in categories of expressing STEM or non-STEM graduate intentions.
The two alternative operationalizations of the dependent variable included a second survey item: students’ degree aspirations in 2008. One alternative of the dependent variable coded students as not intending to pursue graduate school if they marked a response on the graduate majors item but also indicated no post-baccalaureate degree aspirations, as the graduate major intention and degree aspiration variables showed an inconsistency. The second alternative dependent variable coded students who had this inconsistency as missing, thus removing them from the analysis. Appendix B demonstrates that our findings are robust for the original dependent variable as well as the two alternative outcome measures; therefore, we report the results of the original dependent variable described in the variables section.
Lastly, this study does not establish direct causal effects associated with participating in an undergraduate research program. Although a randomized design would better determine causation, our study aimed to address the general impact of a broader set of programs rather than more direct effects of a specific program. Conducting experiments on a larger scale, which includes multiple programs at different institutions, was not feasible. To address the nonrandom nature of our data, we attempted to reduce potential selection bias and also conducted a series of sensitivity analyses to examine how changes in model specification relate to changes in the observed “effect” of participation in undergraduate research on students’ graduate school intentions. Future studies should consider conducting smaller scale randomized experiments on the impact of specific undergraduate research programs, which might also better capture programmatic differences in shaping STEM students’ postgraduate intentions.
Results
Table 1 presents the delta-p statistics for the estimated effect of undergraduate research participation on students graduate school aspirations, and this table includes five estimates: the average treatment effect (ATE), the average treatment for the treated (ATT) effect, the average treatment for the untreated (ATU) effect, an unadjusted model with only the covariates used to calculate the propensity score, and a richer set of covariates that include college experiences, which were measured at the same time we collected data on students’ participation in an undergraduate research program. Examining the first column of delta-p statistics indicates that participation in an undergraduate research program significantly improved students’ probabilities of reporting intentions to pursue a STEM-related graduate program, as the ATE of research participation associated with this graduate school intention was 14.50 percentage points. The delta-p values for ATT (16.56 percentage points) and ATU (14.30 percentage points) are not significantly different from the ATE estimate. Likewise, the delta-p statistic for the unadjusted model with pre-college and college entry variables is not statistically different from the estimates using propensity score weights (delta-p = 17.73), suggesting that a more straightforward, single model can produce estimates that will be similar to parameters obtained from models adjusted by propensity scores.
Table 1. Estimated Treatment Effects for Research Participation on Each Category of the Graduate School Enrollment Intentions Relative to No Plans for Graduate School.
| Intend to Enroll in a STEM | Intend to Enroll in a non-STEM | |||||||
|---|---|---|---|---|---|---|---|---|
| Graduate/Professional Program | Graduate/Professional Program | |||||||
|
|
||||||||
| Delta-P | Log odds | S.E. | Sig. | Delta-P | Log Odds | S.E | .Sig. | |
| Average treatment effect (ATE) | 14.50% | 0.73 | 0.10 | *** | 2.44% | 0.11 | 0.15 | |
| Average treatment for the untreated (ATU) | 14.30% | 0.72 | 0.15 | *** | 2.88% | 0.13 | 0.16 | |
| Average treatment for the treated (ATT) | 16.56% | 0.85 | 0.11 | *** | 0.67% | 0.03 | 0.12 | |
| Unadjusted model (pre-college covariates) | 17.73% | 0.92 | 0.13 | *** | 2.44% | 0.11 | 0.14 | |
| Unadjusted model (all covariates) | 9.04% | 0.44 | 0.17 | ** | 1.77% | 0.08 | 0.15 | |
p < 0.05
p < 0.01
p < 0.001
Including the richer set of covariates reduces the estimated relationship between undergraduate research participation and STEM graduate degree aspirations by almost half (delta-p = 9.04). This finding suggests that undergraduate research participation is connected with several other college experience measures included in the final multinomial HGLM, such as interacting with teaching assistants, joining a professional or departmental club, college GPA, and faculty mentorship. The results in Table 1 indicate that undergraduate research participation does not improve initial STEM students’ probabilities aspiring to non-STEM graduate programs, which suggests that these undergraduate research programs are uniquely tailored to promote interests in STEM graduate programs rather than develop interests in graduate school generally.
Table 2 provides the results of the HGLM for the full set of pre-college, college entry, and college covariates. Findings indicate that Latino students were 7.20 percentage points more likely than their White peers to intend to enroll in STEM graduate or professional programs, and they were 6.27 percentage points more likely to report intentions for non-STEM post-baccalaureate degrees. Black and Asian American students had significantly higher probabilities than White students to plan to pursue a STEM-related graduate or professional program. Mother’s education also positively predicted the likelihood of students’ intentions to pursue a STEM graduate program, as students who came from families where the mother had more formal education had significantly higher probabilities of reporting intentions for a post-baccalaureate degree in STEM.
Table 2. Results of the Multinomial HGLM Analyses Predicting Graduate and Professional School Intentions with All Covariates.
| STEM Graduate Intentions | Non-STEM Graduate Intentions |
|||||||
|---|---|---|---|---|---|---|---|---|
|
|
||||||||
| Coef. | S.E. | Sig. | Delta-P | Coef. | S.E. | Sig. | Delta-P | |
| Research Participation | ||||||||
| Participated in an undergraduate research program | 0.44 | 0.17 | ** | 9.04% | 0.08 | 0.17 | ||
| Background Characteristics and Pre-College Preparation | ||||||||
| Race: Native American | 0.09 | 0.25 | 0.03 | 0.21 | ||||
| Race: Latino | 0.34 | 0.16 | * | 7.20% | 0.29 | 0.14 | * | 6.27% |
| Race: Black | 0.44 | 0.18 | * | 9.12% | 0.25 | 0.17 | ||
| Race: Asian American/Pacific Islander | 0.44 | 0.17 | * | 8.98% | 0.22 | 0.15 | ||
| Sex: Female | −0.13 | 0.14 | 0.04 | 0.14 | ||||
| Parental income | −0.05 | 0.03 | 0.01 | 0.02 | ||||
| Mother’s education | 0.08 | 0.04 | * | 1.63% | 0.06 | 0.03 | ||
| High school GPA | −0.02 | 0.05 | −0.04 | 0.05 | ||||
| Years of high school study in math | −0.15 | 0.12 | −0.18 | 0.11 | ||||
| Composite SAT scores (100) | 0.02 | 0.05 | 0.02 | 0.05 | ||||
| Initial Aspirations and Dispositions | ||||||||
| Came to college to prepare for graduate/professional school |
0.47 | 0.11 | *** | 9.21% | 0.25 | 0.10 | ** | 5.27% |
| Degree aspiration in 2004: Ph.D. | 0.59 | 0.14 | *** | 12.05% | 0.42 | 0.14 | ** | 9.01% |
| Degree aspiration in 2004: M.D., D.O., D.D.O., D.V.M. | 0.68 | 0.15 | *** | 13.90% | 0.19 | 0.16 | ||
| Major in 2004: Life sciences | −0.24 | 0.22 | 0.22 | 0.22 | ||||
| Major in 2004: Engineering/Computer Science | −0.87 | 0.24 | *** | −20.00% | 0.69 | 0.22 | ** | 14.43% |
| Major in 2004: Physical sciences | −0.28 | 0.25 | 0.71 | 0.28 | ** | 14.10% | ||
| Major in 2004: Health professions (excluding nursing) | 0.19 | 0.22 | 0.41 | 0.22 | ||||
| Interactions with Peers and Faculty | ||||||||
| Frequency: Interacted with graduate students/TAs | 0.24 | 0.10 | * | 5.00% | 0.05 | 0.09 | ||
| Joined a pre-professional or departmental club | 0.15 | 0.11 | 0.04 | 0.11 | ||||
| Hours per week spent studying | 0.10 | 0.04 | ** | 2.15% | 0.00 | 0.04 | ||
| Faculty mentorship construct | 0.18 | 0.07 | ** | 3.71% | 0.23 | 0.06 | *** | 5.01% |
| Academic major GPA | 0.17 | 0.04 | *** | 3.63% | 0.03 | 0.04 | ||
| STEM major in 2008 | 2.23 | 0.14 | *** | 48.51% | −1.07 | 0.13 | *** | −22.47% |
| Senior Year Dispositions | ||||||||
| Career focus: Working for social change | 0.14 | 0.08 | 0.37 | 0.07 | *** | 7.70% | ||
| Career focus: Discovery/enhancement of knowledge | 0.27 | 0.07 | *** | 5.66% | −0.03 | 0.07 | ||
| End-Of-College Finances | ||||||||
| Total loans taken out for undergraduate education ($1,000) | 0.00 | 0.00 | −0.01 | 0.00 | ** | −0.17% | ||
| Financial aid: Own resources | −0.02 | 0.04 | −0.05 | 0.03 | ||||
| Financial aid: Aid that does not need to be repaid | 0.04 | 0.03 | 0.03 | 0.03 | ||||
| Financial aid: Aid that must be repaid | −0.03 | 0.03 | 0.02 | 0.03 | ||||
| Institutional Characteristics | ||||||||
| Intercept | −4.11 | 0.76 | 0.84 | 0.64 | ||||
| Control: Private | 0.06 | 0.21 | 0.08 | 0.21 | ||||
| HBCU | 0.00 | 0.48 | 0.28 | 0.41 | ||||
| Selectivity (100) | 0.20 | 0.08 | ** | 4.26% | 0.14 | 0.08 | ||
| Percent White (10) | −0.04 | 0.04 | 0.00 | 0.03 | ||||
| Percent STEM (10) | −0.04 | 0.04 | −0.05 | 0.05 | ||||
| Undergraduate full-time equivalent (log) | 0.13 | 0.11 | 0.10 | 0.11 | ||||
p < 0.05.
p < 0.01
p < 0.001
Source: HGLM analysis of the 2004 CIRP Freshman Survey and 2008 College Senior Survey data. Delta-p statistics are calculated only for significant log odds coefficients.
Students who came to college with aspirations for a medical degree had a 13.90 percentage-point higher probability of reporting plans for a STEM-related graduate or professional program whereas respondents who entered college with Ph.D. aspirations had an increased likelihood of reporting intentions to enroll in any graduate program, regardless of discipline. Students who came to college with aspirations for a Ph.D. were 12.05 percentage points and 9.01 percentage points more likely to report plans for STEM and non-STEM graduate degree programs, respectively. The reference group for freshman-year degree aspirations included bachelor’s degrees, master’s degrees, and J.D.s.
Students who intended to major in engineering when they entered college were 14.43 percentage points more likely to have plans for a non-STEM graduate or professional program and were 20 percentage points less likely to have plans for a STEM graduate degree. This may reflect that many engineering students enter industry immediately after they acquire their bachelor’s degree and/or they pursue advanced degrees in other areas such as business/management. Likewise, students with initial aspirations for physical science majors were significantly more likely to report plans for a non-STEM graduate or professional degree.
Spending more time interacting with graduate students and TAs during college had a significant and positive association with students’ likelihood of reporting intentions to pursue a graduate program connected with STEM disciplines but had no relationship with plans for a non-STEM graduate program. Joining a club related to the academic major did not have a significant association with students’ intentions to pursue STEM or non-STEM graduate programs; however, students who spent more time studying each week also had significantly higher probabilities of reporting intentions to enroll in a STEM graduate or professional program. By contrast, students who felt supported by their faculty were significantly more likely to indicate plans for graduate school regardless of discipline.
High-achieving students had significantly better odds of planning to enroll in graduate or professional programs of any discipline. For every one-point increase from the mean in the academic major GPA variable, students experienced a 3.63 percentage-point increase in their probability to report plans for a STEM-related graduate or professional program. By contrast, students who persisted in STEM disciplines were more than 22 percentage points less likely to report plans for non-STEM post-baccalaureate degrees compared to no graduate school intentions. STEM persisters were nearly 50 percentage points more likely than those who left STEM during college to report plans for STEM graduate or professional programs relative to no graduate school plans.
Respondents’ career focus also had significant associations with their graduate school plans. Students who attributed greater importance to careers that allowed for the discovery and enhancement of knowledge were nearly six percentage points more likely to intend to enroll in a STEM graduate program; by contrast, this characteristic did not significantly predict plans for non-STEM post-baccalaureate degrees.
Finally, students’ finances at the end of college had no substantial relationship with their plans to pursue graduate or professional school. Undergraduate debt and the amounts of various resources used to fund their undergraduate education had no direct association with students’ likelihood to intend to pursue STEM graduate and professional degrees. By contrast, undergraduate debt was significantly associated with lower probabilities of reporting plans for non-STEM graduate study; for every $1,000 increase from the mean in undergraduate debt, students were 0.17 percentage points less likely to report plans for non-STEM post-baccalaureate degrees. This difference could be related to the fact that many STEM graduate programs offer full funding and fellowships in comparison to non-STEM graduate programs that often require students to seek out loans and other forms of financial aid.
The results in Table 2 also provide estimates for the contextual effects of the undergraduate institution that students attended. Students who enrolled in more selective colleges and universities were significantly more likely to report plans to pursue a STEM-related graduate or professional degree. For every 100-point increase in the average composite SAT scores of incoming freshmen, students’ probability of reporting STEM graduate intentions increased by 4.26 percentage points. Institutional selectivity did not significantly predict plans for a non-STEM graduate school enrollment. No other institutional characteristics had a significant association with STEM or non-STEM graduate school plans.
Finally, we tested several student-level and cross-level interaction terms to examine whether the benefits of participation in an undergraduate research program on STEM graduate and professional degree intentions differed across subgroups of students or institutional contexts. Consideration of cross-level interaction terms revealed that the positive relationship between undergraduate research program participation and STEM graduate degree intentions remained consistent across institutions. Student-level interaction terms that crossed research participation with race, gender, initial degree aspiration, and academic major were tested in the model; however, these terms were both non-significant and did not improve overall model fit. Thus, the final model presented in Table 2 excludes any student-level or cross-level interaction terms.
Discussion and Conclusion
Results from this study support previous findings regarding the benefits of undergraduate research participation (e.g., Barlow & Villarejo, 2004; Bauer & Bennett, 2003; Laursen et al., 2010; Lopatto, 2004). Specifically, we found that, even after reducing possible self-selection bias among participants in undergraduate research programs, initial STEM aspirants who gained research experience through these programs were significantly more likely to indicate intentions to pursue a graduate or professional degree in a STEM-related discipline compared to their peers who did not participate in these research programs. Since participation in undergraduate research programs does not appear to significantly affect STEM aspirants’ intentions to pursue graduate or professional degrees in non-STEM fields, it suggests that they participate in programs that are structured in ways that specifically enhance aspirations for STEM degrees—as the ultimate measure of the success of such programs are in increasing the number of science graduates who go on to become scientists.
In using more robust statistical techniques to analyze a larger sample of students and institutions and more rigorous controls such as reports of faculty support and retention in STEM for four years, we found slightly more conservative estimates of the effect of undergraduate research program participation on graduate enrollment intentions than previous studies; however, many of those studies measured actual enrollment in graduate degree programs. Prior studies typically used descriptive statistics and retrospective data from alumni to compare graduate enrollment rates of research participants with those of non-participants and found enrollment differentials that ranged from 17% to 21% (Bauer & Bennett, 2003; Hathaway et al., 2002).
Our results suggest that such participation provides a modest benefit to students, enhancing the likelihood of intending to pursue a graduate or professional program in STEM by approximately 14 to 17 percentage points. Although this effect was reduced in the unadjusted (i.e., non-propensity-weighted) model with the full set of pre-college, college entry, and college experience covariates, the fact that these programs still significantly contribute to intentions to enroll in STEM graduate programs (over and above faculty support and retention in science) suggests there is a unique added value of these experiences that cannot be explained away by other college activities. It is possible that these programs allow students to begin to see themselves as scientists and to perform as scientists (Seymour et al., 2004), which represent two key components of developing a healthy science identity (Carlone & Johnson, 2007). Moreover, undergraduate research experiences provide students with access to faculty mentorship and offer them the opportunity to engage with faculty and peers in scientific discourse. Importantly, however, our study examines students’ intentions to enroll in graduate school rather than actual enrollment; future research should consider applying similar statistical techniques to actual graduate or professional school matriculation data.
Our findings also suggest that using propensity score matching techniques did not provide significantly different estimates of the effect of undergraduate research participation on STEM graduate school intentions when compared with a more straightforward analysis that used the same covariates that were included in estimating propensity scores. This finding supports work by Shadish et al. (2008) who found that more standard estimation techniques that included rich sets of covariates that extended beyond demographic characteristics performed just as well as propensity score models. Thus, in cases where researchers have access to variables salient in predicting differences between cases in the treatment and control conditions, a more straightforward model estimating treatment effects, rather than a propensity score analysis, may be warranted. This more straightforward model can offer additional flexibility in testing interaction effects between two individual-level variables as well as between a contextual and individual variable.
In addition to the effects of undergraduate research, several background characteristics and college experiences had significant associations with the outcome variable. As suggested by status attainment theory, race has a significant influence on students’ desires to pursue graduate school. By the fourth year of college, Latino students appear to have higher aspirations for STEM and non-STEM graduate programs, relative to having no intentions for graduate or professional school, than their White counterparts. Likewise, Black students are significantly more likely than their White peers to report plans for STEM graduate and professional programs; however, previous research suggests that both Black and Latino students pursue graduate programs, regardless of discipline, at lower rates than White students (NSF, 2009). Our findings related to Black and Latino students’ higher degree aspirations supports previous research by Carter (1999, 2001). Thus, while the interest and predisposition for advanced education exist among Black and Latino students, these students’ execution of these intentions is lacking relative to White and Asian American students.
This disconnect points to the likelihood that Latino and Black students have a greater propensity for false positives – indicating advanced degree aspirations without following through on those intentions. The reason for these false positives is unclear. It may be that underrepresented racial minority students have unrealistic degree expectations, as suggested by Astin (1993). By contrast, it may be that the college environment is not equipping Black and Latino students with the skills and resources necessary to successfully act upon their aspirations. Further research is needed in this area.
Although prior research suggests that undergraduate research programs provide students with a mentoring experience where they can connect more meaningfully with faculty (MacLachlan, 2006), our measure of research participation did not directly assess the presence of faculty support or mentorship within the program. Our final multinomial HGLM, however, included a measure of students’ reports of faculty mentorship and guidance, and findings indicate that students who have stronger faculty support also tend to have higher probabilities of reporting plans for either a graduate STEM program or a non-STEM post-baccalaureate degree, which supports the results of previous studies (Carter, 2002; Maton, Hrabowski, & Schmitt, 2000). Feeling supported and mentored by faculty seems to represent an important type of social network that provides students with the guidance necessary for making decisions about post-baccalaureate study (Coleman, 1988, 1990). Likewise, more frequently interacting with graduate students and teaching assistants positively predicted students’ likelihood of reporting plans for a STEM-related graduate or professional degree. Thus, whether originating from faculty or graduate students, higher levels of mentorship and support appear to shape students’ goals and expectations regarding their education.
The findings of this study add to the literature regarding the effect of structured undergraduate research programs on undergraduate and post-baccalaureate outcomes in several important ways. The study utilized a multifaceted approach that included propensity score matching and multinomial HGLM analyses in an attempt to account for observed differences between program participants and non-participants as well as alternative explanations as to why undergraduate research participation leads to STEM graduate and professional degree aspirations. By contrast, previous studies have applied less-robust statistical methods, used limited controls for alternative explanations, and were limited to data collected from a single institution. With these strengths, we can make stronger claims about a relationship between STEM students’ participation in an undergraduate research program and their likelihood of further pursuing graduate studies in STEM.
Most importantly, more students stand to benefit from these programs, resulting in gains in terms of the production of domestic scientific expertise and the diversification of the scientific workforce. Descriptive analyses revealed that just 20% of respondents participated in a structured undergraduate research program during college. This relatively low participation rate begs the question of who has access to these opportunities. Institutional agents (faculty and administrators) must continually assess their programs for such disparities across race, gender, and socioeconomic lines (including first generational status) and find ways to achieve equitable representation among participants. Having disproportionate access to the significant benefits of research programs for certain groups may increase existing disparities along STEM pathways.
By participating in these programs, students receive further encouragement to pursue advanced studies in STEM. Put another way, these research programs may be said to enhance the academic socialization of participants and subsequently further develop their science identities, which lead to an increased likelihood of pursuing advanced STEM studies. Indeed, research by Laursen et al. (2010) and Seymour et al. (2004) claims that undergraduate research programs provide students with both additional tools to navigate the decision process related to graduate school enrollment and the necessary set of skills to make successful graduate school applications. The findings suggest that these structured undergraduate research programs are wise investments for governmental and private agencies and institutions that strive to contribute to the larger goal of sustaining our nation’s capacity to flourish in the areas of science and technology.
Acknowledgments
This study was made possible by the support of the National Institute of General Medical Sciences, NIH Grant Numbers 1 R01 GMO71968-01 and R01 GMO71968-05 as well as the National Science Foundation, NSF Grant Number 0757076. This independent research and the views expressed here do not indicate endorsement by the sponsors.
Appendix A Table of Measures
| Variables | Coding |
|---|---|
| Dependent Variable | |
| Intentions for graduate school | 0=STEM graduate/professional program, 1=non-STEM graduate/professional program, 2=no graduate/professional school intentions |
| Student-Level Variables | |
| Race: Native American | 1=yes, 0=no |
| Race: Latino | 1=yes, 0=no |
| Race: Black | 1=yes, 0=no |
| Race: Asian American | 1=yes, 0=no |
| Sex: Female | 1=yes, 0=no |
| Composite SAT score | Continuous, range from 4 to 16; includes ACT score conversion |
| High School GPA | 1=D to 8=A or A+ |
| HS years of studying math | 1=None to 7=Five or more |
| Income | 1=Less than $10,000 to 14=$250,000 or more |
| Mother’s education | 1=grammar school or less to 8=graduate degree |
|
2004 degree aspiration: Ph.D. (J.D., master’s degree, and bachelor’s degree comprise the reference group) |
1=yes, 0=no |
| 2004 degree aspiration: M.D. (J.D., master’s degree, and bachelor’s degree comprise the reference group) |
1=yes, 0=no |
| Came to college to prepare for graduate/professional school | 1=Not important to 3=Very important |
| 2004 major: Life sciences (nursing is the reference group) | 1=yes, 0=no |
| 2004 major: Physical sciences | 1=yes, 0=no |
| 2004 major: Health sciences | 1=yes, 0=no |
| 2004 major: Engineering or computer science | 1=yes, 0=no |
| Participated in an undergraduate research program | 1=yes, 0=no |
| Joined a pre-professional or departmental club | 1=yes, 0=no |
| Interacted with graduate students/TAs | 1=Not at all to 3=Frequently |
| Career Focus: Working for social change | 1=Not important to 4=Essential |
| Creer Focus: Discovery/enhancement of knowledge | l=Not important to 4=Essential |
| Hours per week spent studying/doing homework | l=none through 8=20 or more |
| Faculty mentorship construct | Construct consisting of seven items of students’ reports of the frequency that faculty provided: encouragement to pursue graduate study; an opportunity to work on a research project; advice and guidance about educational program; emotional support and guidance; a letter of recommendation; feedback about academic work; help in achieving professional goals |
| Academic major GPA | 1=D through 8=A or A+ |
| STEM major in 2008 | l=yes, 0=no |
| Total loans taken out for undergraduate education | Continuous |
| Financial aid: Own resources | 1=None to 6=$10,000 or more |
| Financial aid: Aid which need not be repaid | 1=None to 6=$10,000 or more |
| Financial aid: Aid which must be repaid | 1=None to 6=$10,000 or more |
| Institutional Characteristics | |
| Control: Private | l=yes, 0=no |
| HBCU | l=yes, 0=no |
| Selectivity | Continuous, range of 4 to 16 |
| Proportion of non-White students | Continuous |
| Proportion of STEM undergraduates | Continuous |
| Undergraduate full-time equivalent (log) | Continuous |
Appendix B Sensitivity Analyses Predicting Non-STEM Graduate or Professional School Intentions
| Original Dependent Variable (N=4,152) |
Alternative 1 (N=4,152) | Alternative 2 (N=3,635) | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Coef. | S.E. | Sig. | Delta-P | Coef. | S.E. | Sig. | Delta-P | Coef. | S.E. | Sig. | Delta-P | |
| Research Participation | ||||||||||||
| Participated in an undergraduate research program |
0.08 | 0.17 | 0.08 | 0.17 | 0.13 | 0.18 | ||||||
| Background Characteristics and Pre-College Preparation | ||||||||||||
| Race: Native American | 0.03 | 0.21 | 0.05 | 0.24 | −0.01 | 0.25 | ||||||
| Race: Latino | 0.29 | 0.14 | * | 6.27% | 0.28 | 0.13 | * | 6.98% | 0.37 | 0.15 | ** | 8.53% |
| Race: Black | 0.25 | 0.17 | 0.36 | 0.20 | 0.34 | 0.20 | ||||||
| Race: Asian American/Pacific Islander | 0.22 | 0.15 | 0.07 | 0.21 | 0.28 | 0.20 | ||||||
| Sex: Female | 0.04 | 0.14 | 0.17 | 0.14 | 0.08 | 0.15 | ||||||
| Parental income | 0.01 | 0.02 | 0.01 | 0.02 | 0.01 | 0.02 | ||||||
| Mother’s education | 0.06 | 0.03 | 0.03 | 0.03 | 0.05 | 0.03 | ||||||
| High school GPA | −0.04 | 0.05 | 0.03 | 0.04 | −0.01 | 0.05 | ||||||
| Years of high school study in math | −0.18 | 0.11 | −0.16 | 0.10 | −0.19 | 0.11 | ||||||
| Composite SAT scores (100) | 0.02 | 0.05 | 0.10 | 0.04 | * | 2.51% | 0.03 | 0.05 | ||||
| Initial Aspirations and Dispositions | ||||||||||||
| Came to college to prepare for graduate/professional school |
0.25 | 0.10 | ** | 5.27% | 0.29 | 0.11 | ** | 7.21% | 0.33 | 0.11 | ** | 7.51% |
| Degree aspiration in 2004: Ph.D. | 0.42 | 0.14 | ** | 9.01% | 0.32 | 0.14 | * | 7.98% | 0.46 | 0.16 | ** | 10.59% |
| Degree aspiration in 2004: M.D., D.O., D.D.O., D.V.M. |
0.19 | 0.16 | 0.11 | 0.14 | 0.22 | 0.16 | ||||||
| Major in 2004: Life sciences | 0.22 | 0.22 | 0.29 | 0.19 | 0.22 | 0.23 | ||||||
| Major in 2004: Engineering/Computer Science | 0.69 | 0.22 | ** | 14.43% | 0.65 | 0.19 | *** | 16.05% | 0.60 | 0.22 | ** | 13.64% |
| Major in 2004: Physical sciences | 0.71 | 0.28 | ** | 14.10% | 0.73 | 0.26 | ** | 17.76% | 0.80 | 0.28 | ** | 17.63% |
| Major in 2004: Health professions (excluding nursing) |
0.41 | 0.22 | 0.18 | 0.19 | 0.32 | 0.23 | ||||||
| Interactions with Peers and Faculty | ||||||||||||
| Frequency: Interacted with graduate students/TAs | 0.05 | 0.09 | 0.07 | 0.08 | 0.09 | 0.10 | ||||||
| Joined a pre-professional or departmental club | 0.04 | 0.11 | 0.12 | 0.09 | 0.07 | 0.11 | ||||||
| Hours per week spent studying | 0.00 | 0.04 | 0.01 | 0.03 | 0.02 | 0.04 | ||||||
| Faculty mentorship construct | 0.23 | 0.06 | *** | 5.01% | 0.17 | 0.06 | ** | 4.37% | 0.24 | 0.07 | *** | 5.55% |
| Academic major GPA | 0.03 | 0.04 | 0.09 | 0.04 | ** | 2.33% | 0.06 | 0.04 | ||||
| STEM major in 2008 | −1.07 | 0.13 | *** | −22.47% | −1.18 | 0.10 | *** | −28.59% | −1.15 | 0.12 | *** | −25.71% |
| Senior Year Dispositions | ||||||||||||
| Career focus: Working for social change | 0.37 | 0.07 | *** | 7.70% | 0.23 | 0.05 | *** | 5.75% | 0.34 | 0.06 | *** | 7.66% |
| Career focus: Discovery/enhancement of knowledge |
−0.03 | 0.07 | 0.14 | 0.06 | 0.08 | 0.07 | ||||||
| End-Of-College Finances | ||||||||||||
| Total loans taken out for undergraduate education ($1,000) |
−0.01 | 0.00 | ** | −0.17% | −0.01 | 0.00 | * | −0.13% | −0.01 | 0.00 | * | −0.15% |
| Financial aid: Own resources | −0.05 | 0.03 | 0.02 | 0.03 | −0.02 | 0.04 | ||||||
| Financial aid: Aid that does not need to be repaid | 0.03 | 0.03 | 0.03 | 0.03 | 0.03 | 0.03 | ||||||
| Financial aid: Aid that must be repaid | 0.02 | 0.03 | 0.06 | 0.03 | 0.04 | 0.03 | ||||||
| Institutional Characteristics | ||||||||||||
| Intercept | 0.84 | 0.64 | −0.21 | 0.58 | 0.05 | 0.72 | ||||||
| Control: Private | 0.08 | 0.21 | 0.32 | 0.18 | 0.32 | 0.23 | ||||||
| HBCU | 0.28 | 0.41 | 0.30 | 0.35 | 0.38 | 0.46 | ||||||
| Selectivity (100) | 0.14 | 0.08 | 0.10 | 0.07 | 0.17 | 0.08 | * | 3.93% | ||||
| Percent White (10) | 0.00 | 0.03 | 0.04 | 0.04 | −0.01 | 0.04 | ||||||
| Percent STEM (10) | −0.05 | 0.05 | −0.02 | 0.05 | −0.07 | 0.06 | ||||||
| Undergraduate full-time equivalent (log) | 0.10 | 0.11 | 0.01 | 0.10 | 0.11 | 0.12 | ||||||
| Research Participation | ||||||||||||
| Participated in an undergraduate research program |
0.44 | 0.17 | ** | 9.04% | 0.49 | 0.15 | ** | 11.94% | 0.51 | 0.18 | ** | 11.02% |
| Background Characteristics and Pre-College Preparation | ||||||||||||
| Race: Native American | 0.09 | 0.25 | 0.08 | 0.22 | −0.04 | 0.25 | ||||||
| Race: Latino | 0.34 | 0.16 | * | 7.20% | 0.35 | 0.15 | * | 8.58% | 0.35 | 0.17 | * | 7.70% |
| Race: Black | 0.44 | 0.18 | * | 9.12% | 0.57 | 0.17 | *** | 13.68% | 0.54 | 0.20 | ** | 11.47% |
| Race: Asian American/Pacific Islander | 0.44 | 0.17 | * | 8.98% | 0.17 | 0.15 | 0.42 | 0.18 | * | 9.02% | ||
| Sex: Female | −0.13 | 0.14 | 0.06 | 0.12 | −0.05 | 0.14 | ||||||
| Parental income | −0.05 | 0.03 | −0.04 | 0.02 | −0.06 | 0.03 | * | −1.33% | ||||
| Mother’s education | 0.08 | 0.04 | * | 1.63% | 0.06 | 0.03 | * | 1.58% | 0.08 | 0.04 | * | 1.71% |
| High school GPA | −0.02 | 0.05 | 0.06 | 0.04 | 0.02 | 0.05 | ||||||
| Years of high school study in math | −0.15 | 0.12 | −0.13 | 0.10 | −0.16 | 0.12 | ||||||
| Composite SAT scores (100) | 0.02 | 0.05 | 0.15 | 0.05 | ** | 3.71% | 0.09 | 0.05 | ||||
| Initial Aspirations and Dispositions | ||||||||||||
| Came to college to prepare for graduate/professional school |
0.47 | 0.11 | *** | 9.21% | 0.38 | 0.11 | *** | 9.14% | 0.46 | 0.12 | *** | 9.63% |
| Degree aspiration in 2004: Ph.D. | 0.59 | 0.14 | *** | 12.05% | 0.50 | 0.12 | *** | 12.22% | 0.65 | 0.14 | *** | 13.95% |
| Degree aspiration in 2004: M.D., D.O., D.D.O., D.V.M. |
0.68 | 0.15 | *** | 13.90% | 0.56 | 0.13 | *** | 13.74% | 0.72 | 0.15 | *** | 15.52% |
| Major in 2004: Life sciences | −0.24 | 0.22 | −0.02 | 0.18 | −0.07 | 0.21 | ||||||
| Major in 2004: Engineering/Comp. Science | −0.87 | 0.24 | *** | −20.00% | −0.68 | 0.20 | *** | −16.84% | −0.75 | 0.24 | ** | −17.70% |
| Major in 2004: Physical sciences | −0.28 | 0.25 | −0.15 | 0.22 | −0.11 | 0.24 | ||||||
| Major in 2004: Health professions (excluding nursing) |
0.19 | 0.22 | 0.13 | 0.18 | 0.29 | 0.21 | ||||||
| Interactions with Peers and Faculty | ||||||||||||
| Frequency: Interacted with graduate students/TAs |
0.24 | 0.10 | * | 5.00% | 0.26 | 0.09 | ** | 6.41% | 0.29 | 0.10 | ** | 6.34% |
| Joined a pre-professional or departmental club | 0.15 | 0.11 | 0.32 | 0.11 | ** | 7.93% | 0.26 | 0.12 | * | 6.04% | ||
| Hours per week spent studying | 0.10 | 0.04 | ** | 2.15% | 0.09 | 0.03 | ** | 2.17% | 0.10 | 0.04 | ** | 2.30% |
| Faculty mentorship construct | 0.18 | 0.07 | ** | 3.71% | 0.16 | 0.06 | 0.21 | 0.07 | ** | 4.69% | ||
| Academic major GPA | 0.17 | 0.04 | *** | 3.63% | 0.22 | 0.04 | *** | 5.31% | 0.21 | 0.04 | *** | 4.52% |
| STEM major in 2008 | 2.23 | 0.14 | *** | 48.51% | 2.06 | 0.12 | *** | 47.28% | 2.18 | 0.14 | *** | 48.53% |
| Senior Year Dispositions | ||||||||||||
| Career focus: Working for social change | 0.14 | 0.08 | −0.01 | 0.07 | 0.11 | 0.08 | ||||||
| Career focus: Discovery/enhancement of knowledge |
0.27 | 0.07 | *** | 5.66% | 0.35 | 0.06 | *** | 8.41% | 0.33 | 0.07 | *** | 6.98% |
| End-Of-College Finances | ||||||||||||
| Total loans taken out for undergraduate education ($1,000) |
0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | ||||||
| Financial aid: Own resources | −0.02 | 0.04 | 0.04 | 0.03 | 0.00 | 0.04 | ||||||
| Financial aid: Aid that does not need to be repaid | 0.04 | 0.03 | 0.05 | 0.03 | 0.05 | 0.03 | ||||||
| Financial aid: Aid that must be repaid | −0.03 | 0.03 | 0.01 | 0.03 | −0.02 | 0.04 | ||||||
| Institutional Characteristics | ||||||||||||
| Intercept | −4.11 | 0.76 | −4.63 | 0.66 | −4.51 | 0.80 | ||||||
| Control: Private | 0.06 | 0.21 | 0.03 | 0.18 | 0.07 | 0.21 | ||||||
| HBCU | 0.00 | 0.48 | −0.12 | 0.42 | −0.09 | 0.51 | ||||||
| Selectivity (100) | 0.20 | 0.08 | ** | 4.26% | 0.14 | 0.07 | * | 3.37% | 0.19 | 0.08 | ** | 4.27% |
| Percent White (10) | −0.04 | 0.04 | −0.02 | 0.04 | −0.08 | 0.04 | * | −1.75% | ||||
| Percent STEM (10) | −0.04 | 0.04 | −0.01 | 0.04 | −0.05 | 0.04 | ||||||
| Undergraduate full-time equivalent (log) | 0.13 | 0.11 | −0.04 | 0.10 | 0.11 | 0.11 | ||||||
Appendix C Comparison of Conditional Variables before and after Adjusting the Sample with Propensity Score Weights
| Unweighted | Weighted | ||||||||
|---|---|---|---|---|---|---|---|---|---|
| Researc Part. |
Non- Part. |
Pct. Bias |
p | Research Part. |
Non- Part. |
Pct. Bias |
Pct. Bias Reduction |
p | |
| Native American | 0.06 | 0.05 | 3.70 | 0.32 | 0.06 | 0.06 | 0.00 | 99.20 | 1.00 |
| Latino | 0.19 | 0.23 | −9.40 | 0.02 | 0.19 | 0.19 | −0.30 | 96.40 | 0.94 |
| Black | 0.19 | 0.17 | 4.10 | 0.28 | 0.19 | 0.19 | −0.40 | 89.90 | 0.93 |
| Asian American | 0.16 | 0.13 | 8.20 | 0.03 | 0.16 | 0.16 | 1.10 | 86.50 | 0.83 |
| Sex: Female | 1.58 | 1.64 | −11.30 | 0.00 | 1.58 | 1.60 | −2.50 | 77.70 | 0.61 |
| Income | 8.69 | 8.44 | 7.80 | 0.04 | 8.69 | 8.70 | −0.30 | 95.70 | 0.95 |
| Mother’s education | 5.71 | 5.29 | 21.70 | 0.00 | 5.71 | 5.74 | −1.50 | 93.00 | 0.75 |
| High school GPA | 7.32 | 6.87 | 39.30 | 0.00 | 7.31 | 7.28 | 3.50 | 91.20 | 0.43 |
| HS years studying math | 6.08 | 5.99 | 16.50 | 0.00 | 6.08 | 6.08 | 0.40 | 97.70 | 0.94 |
| Composite SAT (100) | 12.78 | 11.84 | 53.90 | 0.00 | 12.78 | 12.74 | 1.70 | 96.80 | 0.72 |
| Summer research participation | 1.19 | 1.11 | 24.00 | 0.00 | 1.19 | 1.19 | 0.00 | 100.00 | 1.00 |
| Came to college to prepare for graduate/professional school |
2.66 | 2.78 | 22.80 | 0.00 | 2.78 | 2.74 | 7.10 | 68.70 | 0.03 |
| Aspiration for a Ph.D. | 0.35 | 0.24 | 26.30 | 0.00 | 0.35 | 0.35 | 0.30 | 98.80 | 0.95 |
| Aspiration for a medical degree | 0.33 | 0.28 | 10.30 | 0.01 | 0.33 | 0.33 | −0.20 | 98.00 | 0.97 |
| Life sciences major | 0.38 | 0.30 | 16.30 | 0.00 | 0.38 | 0.38 | 0.70 | 96.00 | 0.90 |
| Engineering/Computer science major |
0.24 | 0.27 | −6.80 | 0.08 | 0.24 | 0.24 | 0.40 | 93.80 | 0.93 |
| Physical sciences major | 0.15 | 0.08 | 21.90 | 0.00 | 0.15 | 0.15 | 0.80 | 96.30 | 0.88 |
| Health sciences major | 0.20 | 0.23 | −7.60 | 0.05 | 0.20 | 0.20 | −0.20 | 97.40 | 0.97 |
Source: Mean comparison test of 2004 CIRP Freshman Survey and 2008 College Senior Survey data before and after statistically adjusting the sample with weights calculated from estimated propensity scores.
Footnotes
STEM majors include the biological sciences, physical sciences, computer science, engineering, mathematics, and the health professions.
Analyses of Baccalaureate and Beyond 92/03 data indicate that White and Asian American students with graduate degree aspirations were 3.47 (p < 0.001) times as likely as their peers to enroll in graduate school within 10 years. Likewise, URM students with graduate degree aspirations at the end of college were 2.60 times as likely as their peers to enroll in graduate school within 10 years of finishing their bachelor’s degree.
Contributor Information
M. Kevin Eagan, Jr., University of California, Los Angeles
Sylvia Hurtado, University of California, Los Angeles.
Mitchell J. Chang, University of California, Los Angeles
Gina A. Garcia, University of California, Los Angeles
Felisha A. Herrera, Oregon State Unviersity
Juan C. Garibay, University of California, Los Angeles
References
- Allen W. The color of success: African-American college student outcomes at predominantly White and historically Black public colleges and universities. Harvard Education Review. 1992;62(1):26–44. [Google Scholar]
- Astin AW. What matters in college?: Four critical years revisited. Jossey-Bass Publishers; San Francisco: 1993. [Google Scholar]
- Barlow AEL, Villarejo M. Making a difference for minorities: Evaluation of an educational enrichment program. Journal of Research in Science Teaching. 2004;41(9):861–881. doi: 10.1002/tea.20029. [Google Scholar]
- Bauer KW, Bennett JS. Alumni perceptions used to assess undergraduate research experience. Journal of Higher Education. 2003;74(2):210–230. doi:10.1353/jhe.2003.0011. [Google Scholar]
- Blau PM, Duncan OD. The American occupational structure. John Wiley; New York: 1967. [Google Scholar]
- Buchmann C, Dalton B. Interpersonal influences and educational aspirations in 12 countries: The importance of institutional context. Sociology of Education. 2002;75(1):99–122. [Google Scholar]
- Burke PJ, Hoelter JW. Identity and sex-race differences in educational and occupational aspirations formation. Social Science Research. 1988;17(1):29–47. [Google Scholar]
- Carlone HB, Johnson A. Understanding the science experiences of successful women of color: Science identity as an analytic lens. Journal of Research in Science Teaching. 2007;44(8):1187–1218. doi:10.1002/tea.20237. [Google Scholar]
- Carter DF. The impact of institutional choice and environments on African-American and White students’ degree expectations. Research in Higher Education. 1999;40(1):17–41. [Google Scholar]
- Carter DF. A dream deferred? Examining the degree aspirations of African American and White college students. Routledge Falmer; New York: 2001. [Google Scholar]
- Carter DF. College students’ degree aspirations: A theoretical model and literature review with a focus on African American and Latino students. In: Smart JC, editor. Higher education: A handbook of theory and research. Agathon Press; Bronx, NY: 2002. doi:10.1007/978-94-010-0245-5_3. [Google Scholar]
- Center for Institutional Data Exchange and Analysis . 1999-2000 SMET retention report. University of Oklahoma; Norman, OK: 2000. [Google Scholar]
- Coleman JS. Social capital in the creation of human capital. American Journal of Sociology. 1988;94(S):95–120. doi:10.1086/228943. [Google Scholar]
- Coleman JS. Foundations of Social Theory. Belknap Press; Cambridge: 1990. [Google Scholar]
- Council of Graduate Schools . Graduate education: The backbone of American competitiveness and innovation. Council of Graduate Schools; Washington, DC: 2007. A report from the Council of Graduate Schools Advisory Committee on Graduate Education and American Competitiveness. [Google Scholar]
- Cruce TM. A note on the calculation and interpretation of the delta-p statistic for categorical independent variables. Research in Higher Education. 2009;50(6):608–622. doi:10.1007/s11162-009-9131-1. [Google Scholar]
- DesJardins SL, McCall BP, Ahlburg DA, Moye MJ. Adding a timing light to the “tool box.”. Research in Higher Education. 2002;43(1):83–114. doi:10.1023/A:1013022201296. [Google Scholar]
- Griffin KA, Muniz MM. The Strategies and Struggles of Graduate Diversity Officers in the Recruitment of Doctoral Students of Color. Equity & Excellence in Education. 2011;44(1):57–76. doi:10.1080/10665684.2011.540961. [Google Scholar]
- Guo S, Fraser MW. Propensity score analysis: Statistical methods and applications. Sage; Los Angeles: 2010. [Google Scholar]
- Hathaway RS, Nagda BA, Gregerman SR. The relationship of undergraduate research participation to graduate and professional education pursuit: An empirical study. Journal of College Student Development. 2002;43(5):614–631. [Google Scholar]
- Hearn JC. Impacts of undergraduate experiences on aspirations and plans for graduate and professional education. Research in Higher Education. 1987;27(2):119–141. [Google Scholar]
- Heller DE. Debts and decisions: Student loans and their relationship to graduate school and career choice. USA Group Foundation; Indianapolis, IN: 2001. [On-line] Available: http://www.luminafoundation.org/publications/debtsdecisions.pdf. [Google Scholar]
- Higher Education Research Institute . Degrees of success: Bachelor’s degree completion rates among initial STEM majors. Higher Education Research Institute; Los Angeles: 2010. [Google Scholar]
- Hirano K, Imbens GW. Estimation of causal effects using propensity score weighting: An application to data on right heart catheterization. Health Services and Outcomes Research Methodology. 2001;2(3-4):259–278. doi:10.1023/A:1020371312283. [Google Scholar]
- Hosmer DW, Lemeshow S. Applied logistic regression. John Wiley & Sons, Inc.; New York: 2004. [Google Scholar]
- Hossler D, Gallagher KS. Studying college choice: A three-phase model and the implications for policy-makers. College and University. 1987;62(3):207–221. [Google Scholar]
- Hunter AB, Laursen SL, Seymour E. Becoming a scientist: The role of undergraduate research in students’ cognitive, personal, and professional development. Science Education. 2007;91(1):36–74. doi:10.1002/sce.20173. [Google Scholar]
- Hurtado S, Cabrera N, Lin M, Arellano A, Espinosa L. Diversifying science: Underrepresented student experiences in structured research programs. Research in Higher Education. 2009;50(2):189–214. doi: 10.1007/s11162-008-9114-7. doi:10.1007/s11162-008-9114-7. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Hurtado S, Eagan MK, Cabrera N, Lin M, Park J, Lopez M. Training future scientists: Factors predicting underrepresented minority student participation in undergraduate research. Research in Higher Education. 2008;49(2):126–152. doi: 10.1007/s11162-007-9068-1. Doi:10.1007/sl 1162-007-9068-1. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Laursen S, Seymour E, Hunter AB, Thiry H, Melton G. Undergraduate research in the sciences: Engaging students in real science. Jossey-Bass; San Francisco: 2010. [Google Scholar]
- Lopatto D. Survey of Undergraduate Research Experiences (SURE): First findings. Cell Biology Education. 2004;3(4):270–277. doi: 10.1187/cbe.04-07-0045. doi:10.1187/cbe.04-07-0045. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Lin N. Social capital: A theory of social structure and action. Cambridge University Press; New York: [Google Scholar]
- MacLachlan AJ. Research and Occasional Paper Series: CSHE.6.06. Center for Studies in Higher Education, University of California; Berkeley: 2006. Developing graduate students of color for the professoriate in science, technology, engineering, and mathematics (STEM) [Google Scholar]
- Malcom LE, Dowd AC. The impact of undergraduate debt on the graduate school enrollment of STEM baccalaureates. The Review of Higher Education. 2012;35(2):265–305. [Google Scholar]
- Maton KI, Hrabowski FA. Increasing the number of African American PhDs in the sciences and engineering: A strengths-based approach. American Psychologist. 2004;59(6):547–556. doi: 10.1037/0003-066X.59.6.547. doi: 10.1037/0003-066X.59.6.547. [DOI] [PubMed] [Google Scholar]
- Maton KI, Hrabowski FA, Schmitt CL. African American college students excelling in the sciences: College and postcollege outcomes in the Meyerhoff Scholars Program. Journal of Research in Science Teaching. 2000;37(7):629–654. doi:10.1002/1098-2736(200009)37:7<629::AID-TEA2>3.0.CO;2-8. [Google Scholar]
- McDonough PM. Choosing Colleges: How social class and schools structure opportunity. SUNY Press; Albany, NY: 1997. [Google Scholar]
- McDonough PM. Structuring college opportunities: A cross-case analysis of organizational cultures, climates, and habiti. In: Torres CA, Mitchell TR, editors. Sociology of education: Emerging perspectives. SUNY Press; Albany: 1998. pp. 181–210. [Google Scholar]
- Millett CM. How undergraduate loan debt affects application and enrollment in graduate or first professional school. The Journal of Higher Education. 2003;74(4):386–427. doi:10.1353/jhe.2003.0030. [Google Scholar]
- Mullen AL, Goyette KA, Soares JA. Who goes to graduate school? Social and academic correlates of educational continuation after college. Sociology of Education. 2003;76(2):143–169. doi:10.2307/3090274. [Google Scholar]
- National Science Foundation, Division of Science Resources Statistics . S&E graduate enrollments accelerate in 2007; enrollments of foreign students reach new high (NSF 09-314) National Science Foundation; Washington, DC: 2009. [Google Scholar]
- Nevill SC, Chen X, U.S. Department of Education . The path through graduate school: A longitudinal examination 10 years after bachelor’s degree (NCES-162) National Center for Education Statistics; Washington, DC: 2007. [Google Scholar]
- Nichols A. Causal inference with observational data. The Stata Journal. 2007;7(4):507–541. [Google Scholar]
- Nichols A. Erratum and discussion of propensity-score reweighting. The Stata Journal. 2008;8(4):532–539. [Google Scholar]
- Pascarella ET. College environmental influences on student educational aspirations. A test of a causal model. Journal of Higher Education. 1984;55(6):751–771. [Google Scholar]
- Pascarella ET, Pierson CT, Wolniak GC, Terenzini PT. First-generation college students: Additional evidence on college experiences and outcomes. Journal of Higher Education. 2004;75(3):249–284. doi:10.1353/jhe.2004.0016. [Google Scholar]
- Perna LW. Understanding the decision to enroll in graduate school: Sex and racial/ethnic group differences. Journal of Higher Education. 2004;75(5):487–527. doi:10.1353/jhe.2004.0032. [Google Scholar]
- Perna L. Studying college access and choice: A proposed conceptual model. Higher education: Handbook of theory and research. 2006;XXI:99–157. [Google Scholar]
- Petersen T. A comment on presenting results from logit and probit models. American Sociological Review. 1985;50:130–131. doi:10.2307/2095348. [Google Scholar]
- Raudenbush SW, Bryk AS. Hierarchical linear models: applications and data analysis methods. Sage Publications, Inc.; Thousand Oaks, CA: 2002. [Google Scholar]
- Rosenbaum PR. Model-based direct adjustment. Journal of the American Statistical Association. 1987;82:387–394. doi:10.2307/2289440. [Google Scholar]
- Rosenbaum PR, Rubin DB. The central role of the propensity score in observational studies for causal effects. Biometrika. 1983;70(1):41–55. doi:10.1093/biomet/70.1.41. [Google Scholar]
- Rosenbaum PR, Rubin DB. Reducing Bias in observational studies using subclassification on the propensity score. Journal of the American Statistical Association. 1984;79(387):516–524. doi:10.2307/2288398. [Google Scholar]
- Rosenbaum PR, Rubin DB. Constructing a comparison group using multivariate matched sampling methods that incorporate the propensity score. American Statistician. 1985;39:33–38. doi:10.2307/2683903. [Google Scholar]
- Russell SH, Hancock MP, McCullough J. The pipeline: Benefits of undergraduate research experiences. Science. 2007;316(5824):548–549. doi: 10.1126/science.1140384. doi:10.1126/science.1140384. [DOI] [PubMed] [Google Scholar]
- Sax LJ. Undergraduate science majors: Gender differences in who goes to graduate school. The Review of Higher Education. 2001;24(2):153–172. [Google Scholar]
- Schneider B, Carnoy M, Kilpatrick J, Schmidt WH, Shavelson RJ. Estimating causal effects: Using experimental and observational designs. American Educational Research Association; Washington, D.C.: 2007. [Google Scholar]
- Seymour E, Hunter AB, Laursen SL, DeAntoni T. Establishing the benefits of research experiences for undergraduates in the sciences: First findings from a three-year study. Science Education. 2004;88(4):493–534. doi:10.1002/sce.10131. [Google Scholar]
- Sewell WH, Haller AO, Portes A. The educational and early occupational attainment process. American Sociological Review. 1969;34(2):82–92. [Google Scholar]
- Shadish WR, Clark MH, Steiner PM. Can nonrandomized experiments yield accurate answers? A randomized experiment comparing random to nonrandom assignment. Journal of the American Statistical Association. 2008;103(484):1334–1344. [Google Scholar]
- Sharkness J, DeAngelo L, Pryor J. CIRP Construct Technical Report. 2010 Retrieved March 1, 2010 from http://www.gseis.ucla.edu/heri/PDFs/technicalreport.pdf. [Google Scholar]
- Solorzano DG. The doctorate production and baccalaureate origins of African Americans in the sciences and engineering. The Journal of Negro Education. 1995;64(1):15–32. doi:10.2307/2967281. [Google Scholar]
- Stage FK, John G, Hubbard SM. Undergraduate institutions that foster Black scientists. In: Frierson HT, Tate WF, editors. Beyond stock stories and folktales: African Americans’ paths to STEM fields (Diversity in Higher Education. Vol. 11. Emerald Group Publishing Limited; 2011. pp. 3–21. [Google Scholar]
- Titus MA. Detecting Selection Bias, Using Propensity Score Matching, and estimating treatment effects: An applications to the Private Returns to a Master’s Degree. Research in Higher Education. 2007;48(2):487–521. doi:10.1007/s11162-006-9034-3. [Google Scholar]
- Tran M. How can students be scientists and still be themselves: Understanding the intersectionality of science identity and multiple social identities through graduate student experiences. (Unpublished doctoral dissertation) University of California; Los Angeles: 2011. [Google Scholar]
- Walpole M. Socioeconomic status and college: How SES affects college experiences and outcomes. The Review of Higher Education. 2003;27(1):45–73. doi: 10.1353/rhe.2003.0044. [Google Scholar]
- Weidman JC. Undergraduate Socialization: A Conceptual Approach. In: Smart JC, editor. Higher Education: Handbook of Theory and Research. V. Agathon Press; New York: 1989. pp. 289–322. [Google Scholar]
- Winston GC. Subsidies, hierarchy and peers: The awkward economics of higher education. The Journal of Economic Perspectives. 1999;13(1):13–36. [Google Scholar]
- Wolf-Wendel LE. Models of excellence: The baccalaureate origins of successful European American women, African American women, and Latinas. The Journal of Higher Education. 1998;69(2):141–186. doi:10.2307/2649204. [Google Scholar]
- Zhang L. Advance to Graduate Education: The effect of college quality and undergraduate majors. The Review of Higher Education. 2005;28(3):313–338. doi:10.1353/rhe.2005.003. [Google Scholar]