The STEM Pipeline: The Role of Summer Research Experience in Minority Students' Ph.D. Aspirations

Abstract

Practical research experience has been seen as an important tool to enhance learning in STEM fields and shape commitment to science careers. Indeed, this was a prominent recommendation of the Boyer Commission. Further, there is evidence this is especially important for minority students. In this paper, we examine the role of practical research experience during the summer for talented minority undergraduates in STEM fields. We focus on the link between summer research and STEM Ph.D. program matriculation. We examine evidence on this question using detailed data on students participating in the Meyerhoff Scholarship Program over a 14 year period at the University of Maryland Baltimore County. Our results provide evidence of strong positive effects of summer research on participation in STEM Ph.D. programs. Further, we show that the effects of summer research vary with the frequency and timing of these experiences. The evidence that educational strategies such as summer research experiences improve academic outcomes of minorities is vital, given concern about the science pipeline in the U.S. and the continuing growth in the racial/ethnic diversity of the college-age population.

Introduction

The underrepresentation of minority students in higher education in Science, Technology, Engineering and Mathematics (STEM) fields is disconcerting. According to the National Center for Education Statistics, although underrepresented minority students accounted for 28 percent of undergraduate enrollment in the U.S. in 2007 (NCES, 2010), only about 17 percent of bachelor’s degrees were awarded to minorities in STEM fields during the same year (National Science Board [NSB], 2010). Things are only a bit better in non-STEM fields, where 19 percent of bachelor’s degrees were awarded to underrepresented minorities during the same year (National Science Foundation [NSF], 2007). While the STEM versus non-STEM difference is small for bachelor’s degrees, the gap becomes markedly larger for graduate degrees. Underrepresented minorities earned about 13 percent of master’s degrees and only 7 percent of doctoral degrees in STEM disciplines, compared to 18 percent of master’s degrees and 15 percent of doctoral degrees in non-STEM majors during the same year (NSB, 2010; NSF, 2007).

These numbers are especially discouraging in light of the demographic changes of the past few decades. Since the mid 1970s, the proportion of the college-age population who are African American, Hispanic or Native American has increased from 16 percent to more than 30 percent today. Demographers predict this number will continue to grow, with minorities comprising almost 40 percent of the college-age population by 2020 (NSB, 2004). These demographic shifts mean that differences in educational attainment between non-Hispanic whites and underrepresented minorities raise issues beyond equity. According to a report issued by the Committee on Prospering in the Global Economy in the 21st Century and the Committee on Science, Engineering and Public Policy, due to the recent demographic shifts in the college-age population, the technological and economic strength of the U.S. will depend on the participation and success of minorities in STEM fields, particularly at the highest levels of education (National Academy of Sciences [NAS], 2007). Regardless of whether one finds the National Academy of Sciences’ case compelling that increasing minority participation in STEM education is vital for ensuring U.S. scientific, technologic and economic development, it is clear that demographic changes will mean that expanding opportunities for underrepresented minorities to study science beyond college will be an important mechanism for continued growth in these areas.

These concerns notwithstanding, the persistence of racial disparities in graduate education in STEM fields continues to raise questions about equity and equal opportunity in higher education. An important part of this problem is the lower rate of persistence for minority undergraduate students in STEM majors. Many factors contribute to these lower persistence rates, including financial concerns and relatively poor pre-college academic preparation (Schneider, 2000; Vetter, 1994; Elliot, Strenta, Adair, Matier, & Scott, 1995; Villarejo & Barlow, 2007; Tyson, Lee, Borman, & Hanson, 2007). Villarejo and Barlow (2007) argued that inadequate academic preparation in foundation science courses leads to poor performance in college science classes which often results in attrition of underrepresented minority students from science fields or college in general. Similarly, Tyson et al. (2007) claimed that racial disparities in college attainment in STEM fields persist because underrepresented minority students take less rigorous science and mathematics courses during high school. In addition, financial constraints often prolong time to graduation for these students lowering their motivation to persist. However, there is evidence that even academically gifted minority students with financial support fail to persist in the science pipeline (Seymour & Hewitt, 1997). Seymour and Hewitt argued that factors such as low levels of academic and social integration in college, anxieties induced by negative stereotypes and discrimination, as well as low expectations and support from faculty and parents can have detrimental effects on the academic outcomes of minority students. Other researchers have found similar evidence and claimed that improvement in these areas is essential to increase the success of minorities in STEM education (Barlow & Villarejo, 2004; Building Engineering and Science Talent [BEST], 2004; Maton & Hrabowski, 2004; Summers & Hrabowski, 2006).

In the last two decades, a number of initiatives and educational enrichment programs have been instituted to retain minorities in the STEM education pipeline.¹ These programs have provided various academic services and financial support. A key component has also been to integrate students into the academic and research missions of colleges, universities, and the scientific community more broadly. The logic of providing academic and financial support is obvious. The solid academic preparation and financial support during early college years improve academic outcomes of underrepresented minority students in STEM fields (Villarejo & Barlow, 2007; Tyson et al., 2007). The purpose of integrating students into research is to enhance students’ learning experiences and increase persistence during college, boost interest in STEM careers, and encourage pursuit of graduate education in STEM disciplines. Such research-based learning has become standard practice, especially in science education, and was prominently recommended by the Boyer Commission Report in 1998 (Merkel, 2001).

Unfortunately, there is a limited supply of solid empirical work on the impact of research experience during college on academic success in the sciences (Seymour, Hunter, Laursen, & Deantoni, 2004). In this paper, we attempt to examine the importance of one potential strategy, summer research internships provided to undergraduate minority students majoring in STEM fields. To do so, we obtained data on students participating in the Meyerhoff Scholarship Program at the University of Maryland Baltimore County (UMBC). The Meyerhoff Scholarship Program is one of several programs in universities nationwide to support and encourage success in STEM fields among underrepresented minority groups. The Meyerhoff program is renowned nationally, both because of its success and because the university’s president has been a high profile champion of the program and pursuit of STEM education by minority students more generally.²

Our intent is not to provide an evaluation of that program. Rather, we use detailed data on participants to conduct multivariate regression analyses to assess whether participation in research experience in the form of summer research internships plays a role in shaping later enrollment decisions about pursuing doctoral education in STEM fields.

The Meyerhoff Scholarship Program was established in 1989. Since then more than 600 participants have earned undergraduate degrees in STEM fields. The program incorporates multiple components³ designed to strengthen the academic skills of minority students⁴ and increase their representation in STEM fields. Most importantly, the program seeks to improve participation of minority students in STEM Ph.D. and M.D./Ph.D. programs nationally. By 2008, 71 alumni from the program had earned Ph.D.s while nearly 300 alumni are currently enrolled in graduate and professional degree programs nationwide (UMBC Meyerhoff Scholarship Program, 2008). In addition, the success of the Meyerhoff program in improving opportunities for minority students in STEM fields has been previously noted. For example, one recent study found that Meyerhoff students are almost five times more likely to graduate or be enrolled in a STEM Ph.D. or M.D./Ph.D. programs compared to students from the “declined” comparison sample (i.e., students who were accepted into the program but declined the offer) (Maton, et al. 2009).

The Meyerhoff Scholarship Program provides students with an apprentice-style summer research internships (SRI) where they have opportunities to apply and extend the lessons from their classrooms. Further, these experiences may serve a role in orienting students to scientific norms and establishing networks which may prove useful for students as they plan for and apply for graduate study. Assessing whether SRI experiences encourage participation of minority students in STEM graduate study may, of course, be useful to Meyerhoff program administrators. Beyond any implications for the Meyerhoff program, evidence from this case is likely to have implications for educators and administrators elsewhere as they search for ways to boost the participation and success of minorities in the science pipeline.

Unlike other program components of Meyerhoff (e.g., program values, program community), SRI experiences can readily be replicated in any college or university setting and serve as a potential policy lever that can be employed to improve the success of minority students in STEM fields. Further, SRI are potentially among the most important components contributing to the Meyerhoff program’s success. Evaluation surveys have been administered periodically to Meyerhoff students asking them about the benefits of different program components⁵. Meyerhoff students consistently rate SRI experiences among the most helpful program components. Other program components that have also been highly rated are: Summer Bridge, financial scholarship, being part of the Meyerhoff Program community, study groups, and staff personal advising (Maton, et al., 2009).

The next section of the paper describes how and why SRI experiences might benefit minority students. We present evidence from the previous studies and discuss how this study has the potential to expand our current knowledge about the role of SRI for the persistence of underrepresented minority students in STEM fields. The third section of the paper describes the objectives of our study. We then explain the data we have collected on Meyerhoff students and the methods we use. Next, we present and analyze our results. In the final two sections of the paper, we elaborate on our findings, discuss limitations and offer suggestions for future research.

Background

The idea that directed research activity can serve as a valuable and integral part of an undergraduate education is not novel. For example, and notably, the Boyer Commission Report recommended education reforms which encourage “learning through inquiry rather than the simple transmission of knowledge” (Boyer Commission on Educating Undergraduates in the Research University, 1998, p. 12). Among the advantages of research-based learning highlighted in the Boyer report is the potential effect of such learning on graduate school matriculation. To that end, the report urged colleges and universities to make research practices available to all talented students in order to enhance their skills to the level that will allow these students to make an easy transition into graduate school.

Prior to the Boyer Commission report, few universities coordinated institution-wide undergraduate research programs.⁶ However, in the past decade colleges and universities have expanded their initiatives to incorporate research in their curricula. In a follow-up report, the Boyer Commission on Educating Undergraduates in Research University reports that among 91 research universities, about 62 percent incorporate research in their science curricula (Boyer Commission on Educating Undergraduates in the Research University, 2001). While the growing number of undergraduate research initiatives in science fields is heartening to those who share the concerns voiced in the Boyer report, little evidence is available about the effectiveness of these programs.

Integrating research activities into academic curricula is thought to be especially important for improving retention of minority students in science disciplines. A number of descriptive (Lopatto, 2004, 2007; Russell et al., 2007) and qualitative (Seymour et al., 2004) studies that examine experiences reported by minority students who participated in research activities have supported this belief. For example, Lopatto (2004, 2007) analyzed students’ evaluations of summer undergraduate research experiences that have been documented in the Survey of Undergraduate Research Experiences (SURE). SURE was administered to 2021 undergraduates representing more than sixty institutions during the summer of 2003, summer and fall of 2004 and spring 2005. Participants in the survey engaged in full-time research activities for a minimum of 6 weeks; more than 15 percent of the sample were underrepresented minority students. Lopatto (2007) used descriptive analyses of data on students’ reported gains in the form of enhanced learning experiences, technical skills, a greater resilience in the face of obstacles and independence. A large share of students reported that research experiences either sustained or improved their interest in postgraduate education (i.e., 93 percent of the sample). Moreover, minority students tended to report higher learning benefits obtained from research participation compared to white, non-Hispanic students (Lopatto, 2004, 2007).

A number of studies confirmed and extended these findings beyond science fields, focusing also on technology, engineering and mathematics (Russell, Hancock, & McCullough, 2007; Seymour et al., 2004). For example, Russell et al. (2007) examined experiences of more than 4000 students who participated in NSF programs⁷ and found evidence that students’ confidence, motivation and interest in STEM research careers increased as a result of summer research participation. Seymour et al., (2004) collected in-depth interviews with 76 rising seniors at four liberal arts colleges who participated in the apprentice-style research activities in physics, chemistry, biology, computer science, engineering, biochemistry, mathematics, or psychology during the summer of 2000. They found evidence that research experiences help students confirm, strengthen and clarify their interest in STEM careers, as well as prepare them for graduate school endeavors. In addition, students participating in research programs reported improvements in communication, collegiality and group work (Seymour et al., 2004). Ishiyama (2002) further expanded these findings by including students in fields other than STEM and suggested that research enhanced students’ analytical and logical skills, which improved their ability to develop ideas and perform complex scientific tasks on their own.

Researchers have also examined the effects of research participation on students’ attitudes and behaviors. For example, Lopatto (2007) found that research increases students’ motivation to learn and provides them with better strategies to become active learners. This was especially important for minority students who are more likely to face academic and cultural isolation and low expectations for performance (Seymour & Hewitt, 1997). Hunter, Laursen, and Seymour (2007) made the case that students who participate in research were not only better prepared but were also more confident in their abilities as scientists.

Although there is evidence that students who participate in research benefit in many ways, it remains unclear why and how these benefits affect students’ decisions about graduate school. Studies have used an applied learning model to explain the relationship between research experiences and graduate school participation. Hunter et al. (2007) believe that apprentice-style summer research experiences represent a “community of practice” model of learning. This is a process of learning in which “newcomers are socialized into the practice of the community (science research) through mutual engagement with, and direction and support from an old-timer” (as cited in Wenger, 1998). During this process, students progress from “independent knowing” (realization that knowledge is not absolute and that it requires independent thinking) to “contextual knowing” where knowledge is formed based on the context in which it is located and its authenticity is judged according to its context. The development of contextual knowing usually does not occur until graduate school, so students who participate in apprentice-style summer research during college reach the contextual stage of knowledge much earlier than students who do not (as cited in Baxter Magolda, 2004). This implies that individuals who participate in summer research may be better prepared for graduate education compared to their counterparts.

In addition, Hunter et al., (2007) suggested that by participating in apprentice style summer research internships, students gained understanding and had an opportunity to exercise the practical and cultural knowledge of the science community. Active participation in the authentic practice of the community under the guidance and expertise of a mentor moved the novice from the “periphery” toward “full membership” in the community helping the novice to gain knowledge and self-confidence to deal with the inherent challenges of STEM research (Hunter et al. 2007, p. 38).

Studies that examine factors influencing students to pursue graduate education focused on the effects of student characteristics such as aspirations, goals and abilities, as well as external factors such as academic and social experiences while in college. However, Ethington and Smart (1986) challenged the idea that graduate school participation is largely determined by students’ background characteristics. They argued that students’ experiences and interactions during college have an even larger impact on graduate school matriculation and outcomes than their background characteristics.

Relevant to this, Hathaway, Nagda, and Gregerman (2002) focused on the idea that research engages students in positive learning environments. They argued that educational practices that facilitate academic and social integration during college increase graduate school participation (Hathaway et al, 2002). Astin’s (1999) theory of involvement suggested that students who are involved in academic activities in college are more likely to be dedicated to their studies and persist in college. Undergraduate research participation encourages students to become involved in academic and social activities that improve their cognitive abilities and satisfaction with college, and thus improves their dedication to schooling.

Factors that positively affect student social and academic integration during college are especially important for minority students whose commitment to academics can be affected by a variety of identity issues. Hathaway et al. (2002) believed that research experiences create learning environments in which students build confidence and dedication to the field of study. In addition, research activities are organized in smaller communities where students are able to have more sustained interaction with faculty. These experiences might help minority students clarify and confirm their interests in science careers. Finally, students may discover career options in STEM that they may not otherwise know about and test whether they are well-suited for STEM careers even before they enter the profession (Pedersen-Gallegos, 2007).

Additionally, Kremer and Bringle (1990) made the point that students can derive benefits from involvement in advanced research activities because they earn recognition from the professional community. They are able to participate in research projects that can result in publishable work and opportunities in the form of scholarships. Further, the research products that culminate from these activities makes participants more likely to apply and be accepted at graduate programs that are highly rated in research productivity (Kremer & Bringle, 1990). Participation in summer research also gives students an opportunity to interact with graduate students who are a great source of information about the graduate school experience.

The empirical literature on undergraduate research experiences does generally confirm positive effects of research on college persistence and completion (Barlow & Villarejo, 2004). For example, Barlow and Villarejo (2004) analyzed the Biology Undergraduate Scholars Program (BUSP) at the University of California, Davis (UCD). BUSP provides minority students with the on-campus research experiences in biological sciences. Barlow and Villarejo (2004) utilized multivariate statistical analyses (i.e., multivariate linear and logistic regression models) and compared more than 300 program participants to about 800 students in the declined sample (i.e., students who were accepted into BUSP but declined the offer) who entered in between 1988 and 1994. They found that participation in research activities improves college completion rates and college GPA in science courses. In addition, they tracked more than 100 BUSP participants who enrolled in between 1988–1994 and compared postgraduate outcomes of these students with non-BUSP participants who graduated from UCD in 1999. The descriptive analysis revealed that greater proportion of BUSP participants pursued graduate study in science compared to university graduates overall. Barlow and Villarejo (2004) attributed increased participation in graduate studies of BUSP participants to enriched on-campus research experiences.

Studies that focus on summer research (outside the academic year) similarly suggest an increase in the likelihood of minority students enrolling in Ph.D. programs resulting from research involvement. Foertsch et al. (1996) analyzed the experiences of participants in the Summer Research Opportunity Programs (SROPs) organized in cooperation with the Committee on Institutional Cooperation (CIC; a consortium of 15 Midwestern research universities). SROPs provided more than 5,000 minority students (i.e., 63 % of participants were African-American, 24% Hispanic and 3% Native American) with the opportunity to participate in summer internships at CIC institutions in between 1986 and 1996. The major goal of SROPs was to improve participation of minority students in STEM graduate school. In the evaluation that was conducted by the Learning through Evaluation, Adaptation, and Dissemination (LEAD) Center from UW-Madison and funded by the Spencer Foundation, both qualitative (i.e., structured open-ended interviews with about 20 universities’ SROP directors, 3 minority college representatives, 20 SROP alumni and 10 faculty mentors) and quantitative methods (i.e., descriptively analyzing the CIC’s exit survey of SROP participants and the pre-existing CIC tracking database that contained information on demographic and academic characteristics of 4,585 minority students who participated in the program) have been conducted to examine the effects of research participation on post college outcomes (Foertsch, Alexander, & Penberthy, 1996). Both descriptive and qualitative analyses confirmed that SROPs have been successful in recruiting participants to CIC graduate schools which was driven in large part by the students’ research experiences. Every other SROP’s minority participant enrolled in graduate school, while more than 20 percent went on to enroll in professional schools.

Bauer and Bennett (2003) examined the effects of summer research experiences for a sample of students at the University of Delaware on Ph.D. participation in STEM disciplines. They used descriptive methods and found that 43 percent of STEM alumni who participated in summer research entered a doctoral degree, compared to 23 percent of alumni who did not engage in summer research experiences. Similarly, Zydney, Bennett, Shahid, and Bauer (2002) analyzed a sample of engineering students at the University of Delaware and found that 35 percent of summer research participants enrolled in STEM Ph.D. programs, while only 8 percent of students without research experiences did.

In sum, the literature reviewed suggests a diverse set of variables that explain the purported relationship between involvement in research and pursuit of STEM graduate study among underrepresented minority students. These variables encompass multiple psychological domains, including the cognitive (self-efficacy, analytic problem-solving skills, internalization of values, identity), affective or motivational (motivation to pursue a STEM career; dedication to one’s studies), behavioral (technical and communication skills), and social (socialization into a community of practice; social and academic integration; role models).

Objectives

The work by Foertsch et al. (1996) is insightful. It is also typical of studies on summer research in that the empirical analysis is descriptive. These studies provide insight into the contours of the relationship between research experience and enrollment in Ph.D. programs. Although they suggest that research participation fosters research interests and subsequently affects graduate school matriculation, the empirical evidence that research leads to enrollment in STEM Ph.D. programs is still quite limited. Seymour et al. (2004), reviewed the comprehensiveness and the quality of evidence of more than 50 articles and evaluation reports that examine the benefits of undergraduate research participation in STEM (i.e., engagement in intensive research activities during summer with the direct supervision of faculty researcher). The authors concluded that “examples of well-designed program evaluations…and research findings upon which sound evaluation strategies might be grounded are rare (p. 493)”. In fact, Seymour et al. (2004) claimed that there exists an abundance of program descriptions, explanation of models, and evaluation efforts, but research studies are scarce. Thus, the idea that research participation improves academic outcomes for STEM students is based on rather limited evidence (Seymour et al., 2004). Unlike the previous studies, in this paper we empirically test the relationship between SRI participation and Ph.D. entry. We build upon the idea that summer research resembles the community of practice model of learning through which minority students can enhance their knowledge and skills, and become integrated into the community of science. The argument is that the benefits that students gain through summer research experiences guide their decisions to enter graduate school in STEM fields. So our first research question is:

• Q1Are Meyerhoff students who participate in summer research internships (SRI) more likely to enroll in graduate school in STEM fields compared to Meyerhoff students who do not participate in SRI?

We hypothesize that students who participate in the SRI experiences are more likely to enroll in graduate school in STEM disciplines. We empirically test whether participation in SRI experiences leads to better graduate school outcomes in STEM fields in order to provide stronger support for this relationship than has been available in previous qualitative analyses.

Beyond this question, we examine whether the frequency and timing of SRI experiences matter. The benefits of applied learning are likely cumulative, but may be most important once a baseline level of theory and principles are mastered. However, a community of practice perspective on research internships suggests the importance of early and frequent engagement in applied learning and socialization. Because research internships can occur more than once during the course of a student’s undergraduate studies, we ask the additional questions:

• Q2Are Meyerhoff students who participate in multiple summer research internships more likely to enroll in graduate school in STEM fields compared to the Meyerhoff students who do not participate in summer research internships?
• Q3Does the timing of summer research experiences during the college career affect the graduate school participation of Meyerhoff students?

There is some evidence that students who engage in summer research activities more often, and later in college, are the most likely to enroll in graduate school in STEM fields. Russell and her colleagues (2007), using a survey methodology, found suggestive evidence that for STEM students, the duration of research experience positively affects graduate school expectations.

However, Russell et al. (2007) did not empirically test this relationship and did not find evidence that research participation leads to positive graduate school outcomes. Villarejo and Barlow (2007) found a correlation between later research participation and persistence to biology degree for the cohort of 1988–1994 BUSP students. However, after researchers increased their sample to cohorts 1988–1999 and excluded college dropouts, the results indicated that research participation at any time during college is correlated with obtaining the biology degree. So, while we hypothesize that the effects of applied research experiences are cumulative, meaning that students with more SRI experiences are more likely to enter STEM Ph.D. programs, we cannot anticipate the relative impact of the timing of research experiences on subsequent graduate school entry. Participation by students in the community of scholarship might be most effective early in students’ college careers, as orientation improves subsequent classroom experiences. Or, applied learning might be most fruitful later in college, once foundations and principles are established. In the next section we discuss our data and explain methods used to address the research questions identified above.

Data and Methods

Data Collection

To study the relationship between participating in SRI experiences and subsequent STEM Ph.D. enrollment, we use administrative, academic and survey data collected on students participating in the Meyerhoff program at UMBC since the beginning of the program in 1989. For all students, we obtained academic records from the university. University records included information on students’ academic performance, such as high school grade point average, and SAT math and verbal scores. We also collected survey data from incoming cohorts of Meyerhoff Scholars since the program’s inception in 1989. These surveys provide data on background characteristics, such as race/ethnicity, gender, parents’ education and occupation, high school research experiences, declared major, degree expectations and research excitement at program entry. We also collected data on post-baccalaureate enrollment from surveys and interviews of program alumni. Information was confirmed (or clarified) in the vast majority of cases through review of college transcripts or by phone calls to graduate and professional school registrar offices.

The data on SRI experiences was obtained from student files maintained by the Meyerhoff staff. Program files include information on activities students participated in during the summer months. Meyerhoff students used their summers to do research, take summer classes, work as counselors to incoming freshman participating in the Meyerhoff program or in Upward Bound programs, or other activities. Our research design compares students who took courses, worked as counselors, or did not participate in any activities during summer to those who participated in summer research internships. We collected information on students’ activities for four consecutive summers.

Our dataset initially included data on more than five hundred students. However, 110 observations were dropped because there was no information on the outcome variable (N=70; 12.5% of the potential sample) or on explanatory variables (N=40; 7.1%). The students who were missing information on the outcome variable either had an unknown graduate status (N=50; 8.9%) or were still enrolled as undergraduates at the time the data were collected (N=20; 3.6%). Overall, almost one third (N=36) of the excluded sample did not participate in SRI experiences during college, compared to about one sixth (N=77) of students in our sample. Our final dataset includes information on fourteen cohorts of students entering the program from 1989 to 2002.

Variables

The dependent variable of interest is a dichotomous indicator of whether a student entered a Ph.D. program or earned a Ph.D. in a STEM field after graduation. About 100 students in the sample were currently enrolled in a Ph.D. program (22.1%), while another 45 had already earned their Ph.D. (10%).⁸ We focus on enrollment in Ph.D. programs as the outcome of interest for two reasons. First, graduate study in many STEM fields involves matriculation directly into Ph.D. programs. Many programs do not offer applied master’s degrees, and offer master’s degrees on the route to the Ph.D., or to students who withdraw from the Ph.D. program. Second, it is at the Ph.D. level that the persistence gap between non-Hispanic, white students and underrepresented minority students is most apparent (Reichert & Absher, 1998). Further, there is some evidence that underrepresented minority students enrolled in STEM graduate education are more likely to be enrolled in fields where master’s degrees are more commonly a terminal degree, such as computer science and engineering (NSF, 2004). For both of these reasons, the role of SRI as a means to encourage minority students to enroll in STEM Ph.D. programs is of particular interest.

To analyze the relationship between SRI experiences and graduate school matriculation in STEM fields we employ three different models. The first model measures the overall relationship between participation in SRI experiences during college and graduate school entry. The independent variable is a dichotomous indicator of whether each student participates in SRI experiences at any time during college. Second, we measure the relationship between the total number of years each student participates in SRI experiences and Ph.D. entry to see whether the frequency of research experiences matter. To do this, we add the number of summers that each student participates in SRI experiences during college. Finally, we use a series of indicator variables as control when testing for the relationship between timing of participation in SRI experiences (i.e., participation in an off-campus summer research internship after the freshman, sophomore, junior and/or senior year) and STEM Ph.D. entry or completion.

To test the effects of research, we control for a variety of background, academic and family characteristics. Research studies suggest that graduate school attendance varies based on students’ racial and family background characteristics. There is ample evidence indicating that racial minority and female students participate in graduate school in science disciplines at lower rates relative to non-minority and male students (Sonnert & Holton, 1996; Perna, 2004). Although the share of females who enroll in graduate school in STEM fields has increased over time (e.g., from 39 percent in 1996 to 43 percent in 2004), females continue to lag behind males in graduate school participation and doctoral degree attainment in STEM disciplines (e.g., women earned 37 percent of doctoral degrees in STEM in 2006) (NSF, 2007). Thus, we control for student demographic characteristics by including binary variables for race/ethnicity (i.e., African American, Asian, and White) and gender. Additionally, research on the importance of family background characteristics such as parent’s education has been mixed. While one set of studies argue that parents’ education has little or no influence on students’ postgraduate educational decisions because students are more likely to be independent after college (Stolzenberg, 1994), others suggest that family background indirectly affects graduate school attendance by enhancing degree aspirations, academic performance, and social integration during college (Ethington & Smart, 1986; Mullen, Goyette & Soares, 2003). Moreover, Mullen et al. (2003) find that the effects of parental education vary by graduate program type having a strong influence on student matriculation in first-professional and doctoral programs, but weaker influence on entry in master's programs, and no effect on MBA programs. We include measures of mothers’ and fathers’ education and occupation, each of which we individually separate in four categories to control for the effect of parents’ attributes on students’ academic outcomes. ⁹

As for academic predictors, we control for pre-college student characteristics (i.e., high school grade point average, SAT math and verbal scores) and college characteristics (i.e., declared major).¹⁰ Consistent with the extant literature, college academic success depends on student capacity at college entrance (Burton & Ramist, 2001; Geiser & Santelices, 2007). There is evidence that high school GPA and SAT scores predict academic outcomes during college which further helps students enter and succeed in graduate school (Mullen et al., 2003). In addition, studies on the effects of college major suggest that students majoring in low paying fields are more likely to continue their education after college than students who choose disciplines with higher earnings. In particular, students majoring in bio-science, math/science, social science, humanities and psychology majors are more likely to enroll in doctoral programs than business majors (Zhang, 2005; Mullen et al., 2003). Zhang (2005) also speculates that majors that are more research oriented (i.e., bio-science and math/science) are more likely to lead to Ph.D. participation. Similarly, Sax (2000) finds that students majoring in physical sciences and biological sciences are most likely to enroll in STEM Ph.D. programs, followed by students in mathematical/computer sciences and engineering majors. Sax argues that students who seek high-paying careers tend to turn away from science disciplines.

We also include a binary variable that indicates whether students participate in any type of on-campus academic research to control for additional research experiences. Many studies suggest that on-campus undergraduate research experiences improve dedication to science and increase graduate school attendance in STEM disciplines (Hathaway et al., 2002; Russell et al., 2007). Finally, because the Meyerhoff program was evolving over the period that we have collected data, becoming stronger and more prestigious, we also include fixed effects for each entering cohort (explained below).

Methods

To analyze the role of SRI experiences on STEM Ph.D. entry, we present descriptive statistics and illustrate the joint density of the outcome and treatment variables. We then estimate two separate models related to our research questions. First, we include measures of SRI experiences, and controls for on-campus academic year research, demographic characteristics, and academic characteristics to control for any observable factors that may be related to graduate school participation.

Second, we add cohort fixed effects to limit the possibility that our estimates of the relationship between SRI experiences and Ph.D. participation are driven by characteristics that are specific to particular cohorts. The Meyerhoff program has been developing a national reputation for excellent undergraduate training in STEM fields. Because this reputation has evolved over the years for which we have data, we want to control for the possibility that any relationship we observe between SRI experiences and post-graduate enrollment decisions is not driven simply by a reputation effect. That is, more students may be participating in SRI experiences and matriculating to STEM Ph.D. programs as the program’s reputation grows. Including cohort fixed effects also limits other potential sources of bias, such as the possibility that some cohorts have students more dedicated to science careers, greater science aptitude, or share similar values about research experience. In effect, our estimation strategy compares students participating in SRI with those not participating, within cohorts.

Despite the extensive controls we employ, one threat to the validity of our design is the possibility that within cohorts there are some student characteristics related to seeking SRI placement that is associated with Ph.D. participation that we fail to account for. For example, there may be academic aptitudes correlated with both SRI experiences and application to graduate school that are not fully measured by academic record and standardized test scores. These might include the ability of students to communicate with academic mentors, which might lead to especially supportive recommendations for SRI experiences, as well as for graduate school. Thus, one might argue that the estimated effects in the models above may not necessarily reflect the impact of the SRI participation because there is a chance that only highly motivated and able students choose to enroll in the treatment (i.e., SRI experiences are recommended, but not compulsory). Hence, students’ enrollment in graduate school in STEM may not necessarily be the result of participation in SRI experiences but the effect of generally higher motivation levels that these students have.

Validity Checks

To limit the possibility that our results are driven by these self-selection issues we perform two types of validity checks. First, we examine the relationship between a variety of pre-college and academic characteristics and participation in SRI. We carry out bi-variate tests of hypotheses that students participating do not differ on important dimensions from those who do not participate in SRI. More specifically, we examine factors related to students’ pre-college academic (i.e., high school GPA, SAT math and verbal scores, declared major) and motivational aptitudes (i.e., high school research experience in science and math, degree expectations, research excitement, and career expectations), as well as parents’ characteristics (i.e., mother’s and father’s education and occupation). Further, we introduce a measure of educational expectations and research excitement at college entry in our analysis to control for potential differences in motivational levels between students who choose to engage in SRI experiences relative to those who do not. In other words, we use the degree expectations and research excitement measures as a proxy for motivation in order to obtain better estimates of the effect of SRI experiences on graduate school matriculation in STEM fields. If the magnitude and significance of SRI coefficients do not change as a result of introducing these controls in our model, we can be more confident that our results do not suffer from self-selection.

As a final strategy to address the possibility that participation in SRI is endogenous, we employ a two-stage instrumental variables model. We first estimate the likelihood that a student participated in an SRI, using as instruments the national and state unemployment rates overall and the state unemployment rate in STEM fields for years during which students are enrolled in the Meyerhoff program. We also use measures of research expenditures by the university and grant awards to university faculty during those years. Since both of these are factors that shape the likelihood that a student applies for or receives an SRI, but neither is likely to be related to student characteristics they satisfy the necessary criteria for instruments (Wooldridge, 2002). The second stage uses only the predicted variation in SRI participation to estimate impacts on STEM Ph.D. enrollment. The 2SLS procedure attempts to circumvent the problem of self selection, since the predicted value of SRI participation excludes the unobserved individual characteristics leading to SRI participation, by construction.

Results

Descriptive Statistics

In Table 1, we summarize the characteristics of students in our sample. In the first column, we present descriptive statistics for the whole sample. In the next columns, we distinguish between those students who subsequently enroll in a Ph.D. program in a STEM field, and those who do not. Only a few students (N=7; 5%) who enter or finish a STEM Ph.D. do not participate in SRI experiences during college, compared to more than one-fifth (N=70; 23%) of students in the non-STEM Ph.D. group. In addition, students who enter or finish Ph.D. programs participate in SRI experiences more frequently compared to non-Ph.D. students. For example, 83 percent of students in the former group participate in SRI experiences two summers or more during college, while only about 57 percent of their non-Ph.D. counterparts do (also see Figure 1). Finally, students who enter or finish a STEM Ph.D. program are more likely to participate in SRI each year during college than those who do not. For example, 77 percent of Meyerhoff students who enter or finish a Ph.D. program participate in SRI experiences after their sophomore and junior years, while only 55 percent of their counterparts do (also see Figure 2). Moreover, individuals who enter or finish a STEM Ph.D. program are also more likely to participate in on-campus research during the academic year (53 percent versus 28 percent).

Table 1.

Descriptive Statistics (Standard Errors)

Variable	Description	Full Sample (N=452)		Students Enrolling in Ph.D. (N=145)		Students NOT Enrolling in Ph.D. (N=307)
Any SRI exper.	Yes	0.83	(0.38)	0.95	(0.22)	0.77	(0.42)
SRI experience:	None	0.17	(0.38)	0.05	(0.22)	0.23	(0.42)
total no. of years	One summer	0.17	(0.38)	0.12	(0.33)	0.20	(0.40)
	Two summers	0.31	(0.46)	0.32	(0.47)	0.30	(0.46)
	Three summers	0.32	(0.47)	0.45	(0.50)	0.26	(0.44)
	Four summers	0.02	(0.15)	0.06	(0.23)	0.01	(0.10)
SRI experience:	Freshman	0.44	(0.50)	0.56	(0.50)	0.38	(0.49)
Timing	Sophomore	0.62	(0.49)	0.77	(0.43)	0.55	(0.50)
	Junior	0.62	(0.49)	0.77	(0.42)	0.55	(0.50)
	Senior	0.17	(0.38)	0.24	(0.43)	0.14	(0.35)
On-campus	Research	0.36	(0.48)	0.53	(0.50)	0.28	(0.45)
Race/Ethnicity	White	0.09	(0.29)	0.10	(0.30)	0.09	(0.28)
	African American	0.83	(0.38)	0.83	(0.37)	0.83	(0.38)
	Asian	0.08	(0.27)	0.07	(0.25)	0.08	(0.28)
Gender	Male	0.49	(0.50)	0.43	(0.50)	0.51	(0.50)
Major	Engineering & CMSC	0.29	(0.45)	0.27	(0.44)	0.29	(0.46)
	Bio/Chem/Phys/Math/ Inds. Science	0.50	(0.50)	0.61	(0.49)	0.45	(0.50)
	Other	0.21	(0.41)	0.12	(0.33)	0.26	(0.44)
Mother’s	Low Skilled	0.13	(0.33)	0.09	(0.29)	0.15	(0.35)
Occupation	Service/Sales/Admin.	0.25	(0.43)	0.23	(0.43)	0.25	(0.43)
	Owner/Exec/Mngr.	0.14	(0.35)	0.12	(0.33)	0.15	(0.36)
	Prof. requiring BA/MA	0.48	(0.50)	0.54	(0.50)	0.45	(0.50)
Father’s	Low Skilled	0.16	(0.36)	0.16	(0.37)	0.16	(0.36)
Occupation	Service/Sales/Admin.	0.14	(0.35)	0.09	(0.29)	0.17	(0.38)
	Owner/Exec/Mngr.	0.25	(0.43)	0.23	(0.42)	0.25	(0.44)
	Prof. requiring BA/MA	0.45	(0.50)	0.52	(0.50)	0.42	(0.49)
Mother’s	HS or less	0.11	(0.32)	0.07	(0.25)	0.13	(0.34)
Education	Trade/some college/AA	0.25	(0.43)	0.24	(0.43)	0.25	(0.44)
	Bachelors degree	0.26	(0.44)	0.27	(0.44)	0.26	(0.44)
	Grad/prof. enrol. or deg.	0.38	(0.49)	0.42	(0.50)	0.36	(0.48)
Father’s	HS or less	0.13	(0.34)	0.08	(0.27)	0.16	(0.37)
Education	Trade/some college/AA	0.26	(0.44)	0.26	(0.44)	0.26	(0.44)
	Bachelors degree	0.21	(0.41)	0.20	(0.40)	0.21	(0.41)
	Grad/prof. enrol. or deg.	0.40	(0.49)	0.46	(0.50)	0.36	(0.480
High School GPA	Mean	3.74	(0.36)	3.79	(0.33)	3.72	(0.37)
SAT scores math	Mean	655.31	(52.96)	664.28	(57.10)	651.07	(50.43)
SAT scores verbal	Mean	626.57	(64.22)	625.24	(64.10)	627.20	(64.37)

Figure 1.

Total Number of Summer Research Experiences: Percentage of Students by STEM Ph.D. Entry

Figure 2.

Timing of Summer Research Experiences: Percentage of Students by STEM Ph.D. Entry

Other Covariates

The vast majority of students in our full sample are African-American (N=375; 83%) which is expected due to the nature of the program. Men and women are nearly equally represented (male: N=221; 49%, female: N=231; 51%), with the smaller fraction of male students enrolling in the STEM Ph.D. program (N=63; 43%). About half of the students in the full sample (N=226; 50%) intended to major in Biological Sciences (N=170), Chemistry, Physics, and Mathematics (N=47) and Interdisciplinary science (N=9), while less than one-third (N=129; 29%) planed to major in Engineering (N=85) or Computer Sciences (N=44). The rest of the sample (N=97; 21%) when asked about their intended major before freshman year reported being Other¹¹ (N=55), no declared major¹² (N=30), or unknown (N=2), while very few declared non-science field (N=10). Similarly, a large fraction of students who major in Biological Sciences, Chemistry, Physics, Mathematics and Interdisciplinary Science enters the STEM Ph.D. program (N=88; 61%). Parents of Meyerhoff students tend to be employed in occupations that require at least a bachelor’s or an advanced degree (48% for mothers (N=218); 45% for fathers (N=203)). Clearly, the students’ parents are highly educated, with many having obtained a professional or graduate degree (38% for mothers (N=170) and 40% for fathers (N=179)).

A similar pattern regarding parents’ educational and occupational levels can be observed for the group of students who enter the STEM Ph.D. program. Finally, on average, Meyerhoff students who enter or complete STEM Ph.D.s have higher GPAs in high school (3.79 vs. 3.72) and higher SAT math scores (664.28 vs. 651.07). However, they have quite similar SAT verbal scores (625.24 vs. 627.20).

Summer Research Experience

Table 2 summarizes students’ rates of participation in summer internships, and provides some indication of where Meyerhoff students undertake SRI experiences. Students participate in summer research internships at about 160 different sites. These include research laboratories at UMBC, other U.S. universities, and international locations. In addition, SRI experiences are also offered at private corporations (e.g., IBM, AT&T, HP, Silicon Graphics Systems, Lockheed Martin, 3M, and Apple), government sites (e.g. NIH, FDA, NIST, NASA, and CIA), and pharmaceutical companies (e.g., Merk, Pfizer, and AstraZeneca).

Table 2.

Percent of Meyerhoff Students Participating in Summer Research Internships, by Academic Year

	Freshman	Sophomore	Junior	Senior
UMBC	0.17	0.12	0.14	0.11
Local Univ.	0.21	0.13	0.11	0.11
Other Univ.	0.16	0.32	0.31	0.32
International	0.05	0.13	0.04	0.03
Government	0.21	0.14	0.11	0.13
Private	0.18	0.15	0.22	0.28
Pharmacies	0.03	0.02	0.06	0.03

Although we collect detailed information on types of SRI experiences in which Meyerhoff students participate, we have no way of assessing the quality of these experiences. Thus, we do not differentiate between types of SRI experiences in our multivariate analyses below. Nevertheless, we report the data in the descriptive form merely to give readers additional information on what these experiences involve. For example, after their freshman year, the Meyerhoff students are most likely to undertake SRI experiences at local universities and government and corporate sites. While placements of students at these locations level off after their sophomore year, participation in internships abroad¹³ and at other U.S. universities increases. Additionally, more than one-fourth of Meyerhoff students continue to be placed in research labs at other U.S. universities after their junior and senior years. Lastly, a number of students are also placed in SRI experiences at corporate sites.

To better understand factors that can shape students’ choices about SRI participation, we analyze pre-college academic and demographic characteristics of these students and compare them to the characteristics of students who did not participate in summer research. By exploring previous research experiences and reported future expectations of research, as well as students’ background and academic characteristics prior to college we can obtain profile of students who are likely to participate in SRI while in college as well as of students who are not. Table 3 reports means of key characteristics for students who participated in SRI experiences and those who did not.

Table 3.

Student Characteristics by SRI Participation

Variable	Description	SRI (N=375)	No SRI (N=77)	t-stat	p-value
HS GPA	Mean	3.76	3.63	3.10***	0.002
	St. dev.	0.35	0.39
SAT verbal	Mean	629.15	614.03	1.89*	0.060
	St. dev.	60.67	78.54
SAT math	Mean	658.35	640.52	2.71***	0.007
	St.dev.	50.48	61.94
Variable	Description	SRI (N=375)	no SRI (N=77)	chi2	p-value > chi2
Major	Biology Science	0.597	0.325	10.48***	0.005
	CMSC & Engineering	0.317	0.260
	Other	0.085	0.416
Mother’s	HS or less	0.104	0.156	4.60	0.204
Education	Trade school/some college/AA	0.237	0.312
	Bachelors degree	0.272	0.208
	Graduate/prof. school enrol. or deg.	0.387	0.325
Father’s	HS or less	0.136	0.117	0.24	0.971
Education	Trade school/some college/AA	0.259	0.273
	Bachelors degree	0.211	0.208
	Graduate/prof. school enrol. or deg.	0.395	0.403
Mother’s	Low skilled	0.117	0.182	8.91*	0.063
Occupation	Service/Sales/Administration	0.227	0.338
	Owner/Executive/Manager	0.144	0.130
	Prof. requiring BA/MA	0.509	0.351
Father’s	Low skilled	0.160	0.143	3.90	0.419
Occupation	Service/Sales/Administration	0.133	0.195
	Owner/Executive/Manager	0.237	0.286
	Prof. requiring BA/MA	0.464	0.377
Variable	Description	SRI	No SRI	chi2	p-value > chi2
HS research exp. in science & math*	None	0.248	0.200	5.06	0.281
	A little	0.178	0.140
	Some	0.233	0.380
	Pretty much	0.133	0.100
	A lot	0.208	0.180
Highest level of edu. expected	Bachelor's	0.011	0.026	6.05**	0.049
	Master's	0.078	0.156
	Doctoral	0.912	0.818
Research excitement**	Not at all accurate	0.022	0.055	6.57	0.160
	Not accurate	0.089	0.127
	Somewhat accurate	0.204	0.291
	Accurate	0.275	0.164
	Completely accurate	0.409	0.364
Career focus on practice***	Not at all accurate	0.061	0.036	4.28	0.370
	Not accurate	0.135	0.236
	Somewhat accurate	0.229	0.200
	Accurate	0.268	0.218
	Completely accurate	0.306	0.309
Career focus on research***	Not at all accurate	0.151	0.259	4.80	0.308
	Not accurate	0.309	0.241
	Somewhat accurate	0.251	0.204
	Accurate	0.174	0.204
	Completely accurate	0.116	0.093

Note:

^*Cohort 3 and Cohorts 5 – 14 (318 observations)

^**Cohorts 3 – 13 (368 observations)

^***Cohorts 3 – 13 (365 observations)

chi2:* < .10; ** < .05; *** < .01;

The table shows that SRI participants differ on a number of dimensions. For example, they are better academically prepared before college. For instance, students who engage in summer research have higher mean high school GPA, SAT verbal and math scores than non-participants. SRI participants are more likely to major in Biology Sciences, Chemistry, Physics, or Mathematics (i.e., about 60 percent versus 33 percent for non-SRI participants). SRI participants also are more likely to have parents with occupations that require higher levels of education.

At the same time there are few differences between these groups on a number of dimensions that might be expected to determine whether or not a student participates in an SRI. In particular, having pre-college research experience in science and math is not a significant predictor of SRI participation. Similarly, expressing a high level of excitement about research does not seem to have any effect on SRI participation. And while those who participate in SRI have higher degree expectations, the same share of both SRI and non-SRI participants also responded positively to the statement that they expected that that their career will focus more on research rather than on practice (i.e., 29 percent of SRI participants and non-SRI participants reported this).

The results of Table 3 make clear the need to control for pre-college and college characteristics that are associated with SRI participation, if we want to get insight into the role of this form of applied learning on Ph.D. matriculation. Our multivariate models do this explicitly, allowing us to learn about the role of SRI participation, conditioning on a host of characteristics. Later, we also model participation in SRI programs directly, to try and limit any remaining selection bias which we cannot account for with our control variables.

Next, we turn to our multivariate models. In Tables 4, 5 and 6 we present the results of probit analyses in which we examine the effects of overall participation, frequency and timing of SRI experiences on subsequent enrollment in STEM Ph.D. programs. The results in Table 4 confirm our first hypothesis that students who participate in the SRI experiences are more likely to enroll in graduate school in STEM disciplines. For example, model 1 suggests that students who participate in SRI experiences are about 25 percentage points more likely to enter graduate school in STEM fields compared to students who do not take on these opportunities. In addition, although the on-campus academic year research is a significant predictor of graduate school participation, the estimated coefficient is smaller (i.e., about 16 percentage points) indicating that the SRI experiences are relatively important, and independently predictive.

Table 4.

Probit Estimates of the Effects of Any Summer Research Experience on Ph.D. or M.D./Ph.D. Entry

		Model 1		Model 2 (Cohort FE)
Any SRI exp.	Student participated in SRI while in college	0.902***	(0.246)	1.142***	(0.245)
On-campus	Research	0.466***	(0.163)	0.382**	(0.119)
Race/Ethnicity	African American	−0.089	(−0.031)	0.214	(0.061)
	Asian	−0.465	(−0.138)	−0.402	(−0.105)
Gender	Male	−0.298	(−0.067)	−0.287*	(−0.086)
High School GPA	Mean	0.029	(0.010)	−0.209	(−0.063)
SAT scores math	Mean	0.003**	(0.001)	0.003*	(0.001)
SAT scores verbal	Mean	−0.003**	(−0.001)	−0.002	(−0.001)
Major	Engineering & CMSC	0.249	(0.087)	0.274	(0.086)
	Bio.sci/Chem./Phys./Math/Inds. Sci.	0.389**	(0.131)	0.479**	(0.144)
Mother's	Service/Sales/Administration	0.324	(0.115)	0.265	(0.084)
Occupation	Owner/Executive/Manager	0.069	(0.024)	0.023	(0.007)
	Prof. requiring BA/MA	0.199	(0.068)	0.104	(0.031)
Father's	Service/Sales/Administration	−0.618**	(−0.180)	−0.659**	(−0.162)
Occupation	Owner/Executive/Manager	−0.322	(−0.104)	−0.236	(−0.068)
	Prof. requiring BA/MA	−0.367	(−0.123)	−0.287	(−0.085)
Mother's	Trade school/some coll/AA	0.273	(0.096)	0.326	(0.104)
Education	Bachelors degree	0.144	(0.050)	0.081	(0.025)
	Grad/prof. school enrl/degree	0.156	(0.053)	0.073	(0.022)
Father's	Trade school/some coll/AA	0.394	(0.140)	0.333	(0.106)
Education	Bachelors degree	0.530*	(0.192)	0.584*	(0.195)
	Grad/prof. school enrl/degree	0.675**	(0.235)	0.681**	(0.214)
Constant		−2.522*		−8.781
Number of obs.		452		452
Log Likelihood		−245.93		−224.67
Pseudo R-squared		0.1329		0.2078

Note: Marginal Effects are reported in parentheses

^*< .10;

^**< .05;

^***.01

Table 5.

Probit Estimates of the Effects of the Total Number of Summer Research Experiences on Ph.D. or M.D./Ph.D. Entry

		Model 1		Model 2 (Cohort FE)
SRI experience:	One summer	0.597**	(0.219)	0.879***	(0.307)
total no. of years	Two summers	0.906***	(0.324)	1.133***	(0.377)
	Three summers	1.123***	(0.399)	1.436***	(0.475)
	Four summers	1.837***	(0.620)	2.310***	(0.724)
On-campus	Research	0.371**	(0.129)	0.285*	(0.088)
Race/Ethnicity	African American	−0.177	(−0.062)	0.127	(0.037)
	Asian	−0.530	(−0.153)	−0.466	(−0.118)
Gender	Male	−0.216	(−0.073)	−0.328**	(−0.098)
High School GPA	Mean	−0.101	(−0.034)	−0.367	(−0.110)
SAT scores math	Mean	0.003**	(0.001)	0.003*	(0.001)
SAT scores verbal	Mean	−0.003***	(−0.001)	−0.002	(−0.001)
Major	Engineering & CMSC	0.239	(0.083)	0.270	(0.084)
	Bio.sci/Chem./Phys./Math/Inds. Sci.	0.395**	(0.133)	0.495**	(0.148)
Mother's	Service/Sales/Admin.	0.337	(0.119)	0.274	(0.086)
Occupation	Owner/Executive/Manager	0.038	(0.013)	−0.032	(−0.009)
	Prof. requiring BA/MA	0.214	(0.072)	0.100	(0.030)
Father's	Service/Sales/Admin.	−0.688**	(−0.195)	−0.735**	(−0.175)
Occupation	Owner/Executive/Manager	−0.303	(−0.097)	−0.212	(−0.061)
	Prof. requiring BA/MA	−0.426	(−0.142)	−0.356	(−0.105)
Mother's	Trade school/some coll/AA	0.271	(0.095)	0.331	(0.105)
Education	Bachelors degree	0.087	(0.030)	0.017	(0.005)
	Grad/prof. school enrl/deg.	0.114	(0.039)	0.051	(0.015)
Father's	Trade school/some coll/AA	0.382	(0.135)	0.324	(0.103)
Education	Bachelors degree	0.526*	(0.190)	0.593*	(0.198)
	Grad/prof. school enrl/deg.	0.699**	(0.242)	0.705**	(0.221)
Constant		−1.592		−7.432
Observations		452		452
Log likelihood		−240.55		−218.47
Pseudo R-squared		0.152		0.223

Note: Marginal Effects are reported in parentheses;

^*< .10;

^**< .05,

^***.01

Table 6.

Probit Estimates of the Effects of the Timing of Summer Research Experiences on Ph.D. or M.D./Ph.D. Entry

Variable	Definition	Model 1		Model 2 (Cohort FE)
SRI experience:	Freshman	0.144	(0.049)	0.083	(0.025)
Timing	Sophomore	0.401**	(0.131)	0.556***	(0.158)
	Junior	0.400***	(0.131)	0.492***	(0.140)
	Senior	0.620***	(0.228)	0.740***	(0.254)
On-campus	Research	0.378**	(0.131)	0.292*	(0.090)
Race/Ethnicity	African American	−0.203	(−0.071)	0.104	(0.030)
	Asian	−0.558*	(−0.160)	−0.515	(−0.128)
Gender	Male	−0.215	(−0.073)	−0.308*	(−0.092)
High School GPA	Mean	−0.074	(−0.025)	−0.344	(−0.103)
SAT scores math	Mean	0.003**	(0.001)	0.003**	(0.001)
SAT scores verbal	Mean	−0.003**	(−0.001)	−0.002	(−0.001)
Major	Engineering & CMSC	0.298	(0.104)	0.329	(0.104)
	Bio.sci/Chem./Phys./Math/Inds. Sci.	0.507**	(0.170)	0.634***	(0.189)
Mother`s	Service/Sales/Admin.	0.340	(0.120)	0.255	(0.080)
Occupation	Owner/Executive/Manager	0.040	(0.014)	−0.027	(−0.008)
	Prof. requiring BA/MA	0.230	(0.078)	0.115	(0.035)
Father`s	Service/Sales/Admin.	−0.688**	(−0.195)	−0.737**	(−0.175)
Occupation	Owner/Executive/Manager	−0.301	(−0.097)	−0.204	(−0.059)
	Prof. requiring BA/MA	−0.377	(−0.126)	−0.286	(−0.085)
Mother`s	Trade school/some coll/AA	0.243	(0.085)	0.292	(0.092)
Education	Bachelors degree	0.041	(0.014)	−0.081	(−0.024)
	Grad/prof. school enrl/deg.	0.071	(0.024)	−0.052	(−0.016)
Father`s	Trade school/some coll/AA	0.414	(0.147)	0.375	(0.120)
Education	Bachelors degree	0.505*	(0.182)	0.567*	(0.189)
	Grad/prof. school enrl/deg.	0.683**	(0.237)	0.704**	(0.220)
Constant		−1.941		−7.751
Observations		452		452
Log likelihood		−239.17		−216.37
Pseudo R-squared		0.157		0.237

Note: Marginal Effects are reported in parentheses;

^*< .10;

^**< .05,

^***.01

In Model 2, we add cohort fixed effects, to control for possible changes in the characteristics of students, the program, or its reputation that might affect SRI placement, and Ph.D. program entry. Upon doing so, our results change little, reassuring us that the coefficients that we obtained in the previous model are not driven by variation of unobservable characteristics over time. In other words, it is unlikely that the program’s reputation effect could have had an effect on the later cohorts. On the other hand, the reputation effect may have been affecting the estimate on the coefficient for the on-campus academic year research. For example, in model 2, the coefficient drops to 12 percentage points, indicating that within cohorts the effect of the on-campus academic research is smaller, possibly due to the reputation effect.

The results in Tables 4, 5, and 6 also indicate that compared to their non-Hispanic white peers, Asian students are less likely to enter Ph.D. programs in STEM fields (see Table 6). However, although the magnitude of the coefficient is large, it is only marginally significant. In addition, this result must be interpreted with caution because only a small fraction of students in our data are Asian (N=22). Male students are less likely to enter STEM Ph.D. programs, but the coefficient is marginally significant only in the cohort fixed effects model. This result is likely due to the fact that there has been an increase in the proportion of females entering graduate school in STEM fields in the past decade. Today, women represent almost half (43 percent) of graduate students in STEM fields. In addition, females tend to be overrepresented in fields such as biological sciences: In 2006, women made up 56 percent of the graduate students in biological sciences, but only 23 percent of graduate students in engineering and 25 percent in computer sciences (NSF, 2007). A large share of students in our sample majored in biological sciences (50 percent), while a smaller proportion chooses engineering or computer science (29 percent). Although women are comparable to men in terms of STEM graduate school participation, there still exists a large gap in the Ph.D. attainment between the two groups in STEM fields (e.g., females earned 37 percent of doctoral degrees in STEM fields in 2004) (Commission on Professionals in Science and Technology [CPST], 2007). Thus, the effect of gender may differ depending on outcome (STEM Ph.D. enrollment versus STEM Ph.D. attainment).

In addition, high SAT math scores increase participation in graduate school, but the magnitude is minimal. On the other hand, high school GPA and SAT verbal scores are not significant predictors of Ph.D. entry in STEM fields (i.e., the coefficient on SAT verbal scores is marginally significant in model 1 in each analysis, but is quite small). As for major, students planning on majoring in Biology sciences, Chemistry, Physics, Math or Interdisciplinary Sciences are more likely to enter Ph.D. programs in STEM fields compared to students in non-science majors or those who are undeclared during their freshman year. Finally, results on parental characteristics indicate students with fathers occupied in low level jobs are less likely to enter STEM Ph.D. programs, while fathers who have high levels of education may positively affect graduate school STEM Ph.D. entry of their children.

In Table 5, we turn to the question of whether the role of SRI participation accumulates over multiple internships. The results from model 1 do indeed suggest that students who participate in SRI experiences more frequently are more likely to enroll in STEM Ph.D. fields. Although any participation in SRI is better than none, the largest estimate is for students who participate in SRI experiences at least three summers. For example, students who participate in SRI experiences for at least three summers are about 40 and 62 percentage points more likely to enroll in graduate school in STEM compared to students who do not participate in SRI experiences. The coefficient is also large (i.e., about 32 percentage points) for students who engage in SRI experiences for two summers. As for the other covariates, the significance level and the magnitudes remain the same except for the on-campus academic year research, which is slightly lower relative to Model 1 in the previous table. It is possible that SRI experiences work as a substitute for on-campus academic year research. In other words, as students engage in more years of SRI experiences, the on-campus research becomes less relevant. Finally, Model 2 in Table 5 indicates that even when cohort effects are included, the SRI coefficients remain unchanged. In fact, the magnitudes increase by a small fraction. This is further evidence that unmeasured, time-varying attributes are not driving our estimates.

In Table 6 we turn to our third research question: Does the timing of summer research experiences during their college career affect graduate school participation of Meyerhoff students? Overall, on average, participation in the SRI experiences during any summer besides the summer after the freshman year is associated with a higher probability that the student enters a Ph.D. program. More specifically, the results in Table 6 suggest that students who engage in SRI experiences after their senior year in college are most likely to enroll in graduate school in STEM fields. There are also positive effects on STEM Ph.D. entry for students who participate in SRI experiences after their sophomore and junior years with the benefits of participating in SRI after sophomore year being equal to gains obtained from participating in SRI experiences after junior year. We estimate that the probability of enrolling in a STEM Ph.D. program is 23 percentage points higher for students who participate in SRI experiences after their senior year than those who do not, while the coefficients on participation in SRI experiences following sophomore and junior years on STEM Ph.D. matriculation are somewhat smaller (e.g., 13 percentage points). In addition, relative to the model 1 in Table 5, other covariates generally remain unchanged. Finally, similar to the results in Tables 4 and 5, when we include cohort fixed effects, coefficients on SRI experiences change little, increasing slightly. Of note is that the magnitude of SRI participation after sophomore year increases to 16 percentage points while junior year remains lower (i.e., 14 percentage points).

In Table 7 we summarize the results of our two supplementary analyses to test the validity of our findings. First, we supplement the fixed effects model in Table 4, by including a measure of research excitement and expectations from a survey administered prior to enrolling as a freshman. To conserve space, we report the coefficient of interest in Table 7 (and its marginal effect): the impact of any SRI participation on subsequent STEM Ph.D. enrollment. The 1.250 coefficient in Table 7 is virtually identical to the 1.142 obtained in Table 4. To the extent that SRI participation is picking up unmeasured aptitude or attraction to research or plans for a research career, this helps limit that concern.

Table 7.

Probit Estimates of the Effects of Any Summer Research Experiences on Ph.D. or M.D./Ph.D. Entry

		Model 1	Model 2
		(Research Excitement and Degree Expectations)	(IV Estimation)
Any SRI experience	Student participated in SRI while in college	1.250** (0.272)	0.069
Observations		367	452
Log likelihood		−167.52
Pseudo R-squared		0.278
F-test:			6.86 (p=0.0000)
Hausman Test:			0.157 (p=0.522)
Test of overidentifying restrictions:			<0.01

Note: Marginal Effects are reported in parentheses; Controls for on-campus academic research, academic characteristics, demographic characteristics, and father’s and mother’s education and occupation included in both models; Model 1 also includes cohort fixed effects.

^*< .10;

^**< .05,

^***.01

In the second column of Table 7, we summarize the results of our 2SLS estimation. We report the 2^nd stage estimate of the effect of SRI participation on STEM Ph.D. entry, as well as an F-test of joint significance for our instruments in the first stage, the Hausman test of endogeneity, and a test of the exclusion restrictions. The point estimate from the second stage is insignificant, suggesting that the contribution of SRI experiences to Ph.D. enrollment might be zero. However, the instruments are quite weak. The F-test of joint significance is below 10, a common standard for predictive power of instruments. The p-value for the Hausman test of endogeneity exceeds 0.10, suggesting that based on our instruments, we can not reject the null hypothesis that SRI are exogenous. Finally, though the instruments are weak, the test of overidentification suggests that they are excludable. Nonetheless, the weakness of the instruments in the first stage, and the finding that our instruments provide no evidence of endogeneity means that the 2SLS procedure here does not quell concern that our results are driven by selection bias, but it does not settle the matter either.

Discussion and Conclusions

Although previous studies have suggested positive effects of summer research experiences on graduate school participation, to date there has been limited empirical support for such claims. To provide direct empirical tests of the relationship between summer research and graduate school outcomes, we employed unique longitudinal data on the MSP participants and estimated three different models to examine the effects of the overall participation, frequency and timing of SRI experiences. Our models included a range of controls for students’ background, academic, and research characteristics, in addition to cohort fixed effects.

We anticipated a positive relationship between SRI experiences and Ph.D. entry because SRI experiences are thought to provide students with additional learning and the opportunity to gain skills and confidence useful for success in graduate school. In addition, through SRI experiences students build relationships with other researchers and learn how to participate and behave in the science community. Further, we speculated that the effects of SRI experiences differ depending on how often and when students engage in these activities. In particular, we expected that more frequent SRI experience during college would increase the probability that students enter graduate school in STEM fields. Finally, we suspected that the effects of SRI experiences are not uniform across years because students may increasingly recognize the benefits of research as they approach the time to apply to graduate school.

Our findings suggest that SRI experiences encourage participation in Ph.D. programs in STEM fields. In addition, the relationship is stronger for students who undertake SRI experiences more often. In particular, our results indicate that students who participate in SRI experiences for more than two summers have high likelihood of entering STEM Ph.D. programs compared to students who do not engage in SRI experiences. One explanation for the “frequency effect” may relate to the previous findings by Russell et al. (2007) who claim that the duration of research experiences has a positive effect on graduate school expectations in STEM. Thus, students who engage in SRI experiences more often spend more time in these activities altogether and in the process develop higher graduate school aspirations. Further, having higher degree expectations has been found to be positively related to graduate school outcomes in STEM, particularly for minorities (Sax, 2000; Pascarella, Wolniak, Pierson & Flowers, 2004). Hence, the frequency of research experiences may affect graduate school participation. In addition, there is some indication that participation in SRI experiences later in college (i.e., after senior year) may be the most beneficial relative to other years regarding the decision to pursue STEM Ph.D. programs. However, although the relationship is not as strong for other years, the fact that there are no differences in the benefits of SRI experiences after sophomore and junior years demonstrates that participation in any year after the freshman year may be essential for STEM Ph.D. entry. In other words, although it is possible that those students who seek research experiences later in their college career are the most serious in terms of preparing for a research career, this does not mean that research experiences earlier in college are not as important.

The results of this study are useful for administrators and program directors at colleges and universities whose goal is to develop strategies to aid minority students in STEM fields, as well as federal government agencies interested in STEM education and careers. Our findings make clear that applied research experience during summers can increase the likelihood that minority students will pursue graduate training in STEM fields. There is good evidence that engaging students in authentic practice help them develop researcher identities by integrating them in the community of practice (Edelson & Reiser, 2006; National Research Council [NRC], 2000). In the apprentice-style learning like that engaged in by Meyerhoff students during summer, students work collaboratively with experts involved in STEM fields. This experience can help students develop “contextual knowledge” during which they build self-confidence and take primary responsibilities for the projects moving from the “periphery” toward “full membership” in the community (Hunter et al., 2007). Furthermore, engaging such learners with authentic practice is critical for them to learn the multiple challenges of being a researcher (Newstetter, Johri & Wulf, 2008).

Universities have traditionally used an “instructionist” teaching model in which the instructor has the relevant knowledge and passes it on to learners. This model has been challenged from many theoretical and practical standpoints. For example, a recent NSF workshop advocated active learning that involves participation in the apprentice-style summer internships and integrates students in communities of practice where they can share knowledge and resources in support of collaborative action (Lorden & Slimowitz, 2003; Fischer, Rohde & Wulf, 2007). Researchers have identified various benefits of building a community among undergraduate researchers along with the evidence that students’ dedication to science careers and persistence increases if they feel that they belong to a supportive group with common interests (Bender, Blockus & Webster, 2008; Lopatto, 2004; Nagda, Gregerman, Jonides, von Hippel, & Lerner, 1998; Seymour, et al., 2004).

We believe our findings are practical and important for higher education administrators and educators who wish to improve the post-graduate and career prospects for minority students in STEM fields. Much of the literature has rightly focused on the experiences of a relatively small number of now-established programs around the country. Some examples include the Meyerhoff Scholarship Program, as well as the BUSP and McNair Scholars Program at UC Davis, and undergraduate research programs at Emory University, Massachusetts Institute of Technology (MIT), and California Institute of Technology (Maton et al., 2000; 2009; Villarejo & Barlow, 2007; Merkel, 2001). We do not focus on the package of interventions employed by many programs. Rather, we examine the impact of one intervention that can be implemented in settings that do not provide a multitude of services. Thus, our findings suggest that administrators and educators can affect minority students’ commitments to STEM careers through programs that encourage and coordinate summer internship placements. This means that interventions that can affect students need not be especially costly, and can leverage resources outside the college or university.

In addition, our findings are relevant for policy makers beyond the institutional level. Given the recent increase in the diversity of the student population, higher education institutions need to engage in innovative teaching approaches, particularly in STEM fields. Participation in undergraduate research that integrates “active” modes of learning and enhances novice-expert relationships has been proposed to be an especially effective educational strategy (NSF, 2003). Our research confirms this conjecture. At a time when student interest in most sciences is declining and the achievement gap between underrepresented minority and non-Hispanic white and Asian students persists, the use of innovative approaches in undergraduate science courses is essential. We believe that replicating the apprentice-style summer research internships is one important strategy that can be used to enhance student learning in higher education.

The consistency of our SRI estimates over two specifications in each of the three models helps establish the reliability of our results. In fact, the models that include cohort fixed effects show that the underlying factors we suspected might have been affecting our estimates (i.e., reputation effect) are not present. Although our results are robust, there are several limitations to the current study, pertinent to both internal and external validity. One of the limitations pertains to the endogeneity problem. That we control for so many covariates measuring preparation and ability helps limit this concern. Further, our specification check provides some additional confidence in these results. Our two-stage estimation procedure, however, did not confirm these results. Our two stage estimates suggest the contribution of SRI to Ph.D. enrollment might be zero. However, the instruments on which those claims are made are quite weak. Indeed, they failed both standard tests of instrument strength. A better, more robust set of instruments could fruitfully test the principal hypothesis.

The program setting and structure also limit the external validity of our findings. Beyond the limitations due to the fact that the program is set at one university, since summer research internships are offered to the most academically gifted, it may be incorrect to assume that effects of the same magnitude would be observed in other settings and across different sets of students. In addition, since the Meyerhoff program includes fourteen different interrelated components, it is not clear whether summer internships offered in other contexts would have the same effects as seen here. The closest example is the BUSP program because it is similar in character to the Meyerhoff Scholarship Program. Even though Barlow and Villarejo’s (2004) findings are consistent with ours, they analyze the effects of on-campus research during the regular academic year as opposed to the summer research experiences that we examine in our study.

While the distinctive nature of the Meyerhoff program makes it hard to compare our results to other similar educational enrichment programs, this should not discount the findings established in the study. Our estimates of the effects of SRI experiences are quite large, so even if other components of the MSP affect our results, it is unlikely that these components can explain all or even most of the effect. In fact, these results should be useful to other researchers who seek to explore the benefits of apprentice-style summer research as a potential mechanism for retaining minorities in the science pipeline.

Nonetheless, an important limitation to our study suggests directions for further research. The theory-based mechanisms that can determine a causal linkage between research experience and pursuit of graduate study must be explored. As noted in the literature review, a diverse set of variables have been suggested to explain the hypothesized relationship between research experience and graduate study. One important set of variables are cognitive, including self-efficacy (Hunter et al., 2007; Russell et al., 2007), analytic problem-solving skills (Ishiyama, 2002) and emerging identify as a scientist (Seymour et al., 2004). A second set of variables are affective/motivational, including motivation to pursue a STEM career (Active Learning Theory, Lopato, 2004, 2007; Russell et al., 2007) and dedication to one’s studies (Astin, 1999). A third set are behavioral, including enhanced technical skills (Lopato, 2004, 2007) and communication skills (Seymour et al., 2004). A final set are social, including socialization into the scientific community (Community of Practice Theory; Hunter et al., 2007) and social and academic integration into the university environment (Theory of Involvement, Astin, 1999; Hathaway et al., 2002). Unfortunately, measures assessing these and related variables were not available in the current data set, so our research could not test the relative importance of different variables and variable domains, the causal interrelationships among them, or the primacy of one theoretical model versus another. A major priority for future research is initiation of longitudinal studies that incorporate reliable and valid measures of theory-based variables. These characteristics must be assessed on multiple occasions during the course of students’ college careers, so that direct examination of various proposed mechanisms of influence of research experience on pursuit of graduate study can be determined.

In this paper, we have attempted to provide empirical evidence concerning the relationship between summer research internships for undergraduate students on subsequent entry into STEM Ph.D. programs. For many reasons, students can expect to benefit from summer research internships. These gains are related to the recommendations from the Boyer Commission’s Academic Bill of Rights, which advocates that research universities give their students “the opportunity to learn through inquiry rather than the simple transmission of knowledge” and should prepare students for “…whatever may lie beyond graduation, whether it be graduate school, professional school, or first professional position” (Boyer Commission on Educating Undergraduates in the Research University, 1998, p. 12). In an era in which the U.S. struggles to remain a global powerhouse in science and technology, increasing the number of students pursuing graduate school in science is crucial. Relying on a range of demographic groups, especially minorities, is a potential solution. Since empirical evidence suggests that summer research opportunities help retain minorities in the science pipeline, educational policymakers should support the development of educational enrichment programs that offer apprentice-style summer research opportunities.

Biographies

Matea Pender is as a research assistant in the Public Policy Department at University of Maryland Baltimore County (UMBC) and an adjunct instructor at Community College of Baltimore County (CCBC). She received a Bachelor’s degree in Information Systems, a Masters in Economic Policy Analysis, and her Ph.D. in Public Policy from UMBC. In May 2010, Dr. Pender successfully completed her dissertation on the effectiveness of educational policies that affect college persistence of students at-risk: first-generation college students, immigrants and underrepresented minorities. Dr. Pender worked on a large-scale, evaluation task with a multidisciplinary team of researchers examining the effectiveness of the Meyerhoff Scholarship program concerned with low participation of underrepresented minorities in science careers. Dr. Pender presented her work about this project at the 2^nd Annual Conference on Understanding Interventions that Encourage Minorities to Pursue Research Careers in Atlanta.

Dave E. Marcotte is Professor of Public Policy at the University of Maryland Baltimore County (UMBC). Dr. Marcotte’s fields of interest include social policy, employment and training policy, education, mental health, and labor economics. His work is largely in the area of applied economics. Prof. Marcotte has worked with state and federal agencies on a variety of projects to evaluate and improve management of public job training programs. He was also involved in the congressionally mandated National Assessment of Vocational Education, as part of a team of researchers from Columbia University and the University of California at Berkeley. He is coeditor of a book, The Economics of Gender and Mental Health, published by Elsevier-Science in 2004. His current research includes various projects on mental health, suicide, and education at community colleges.

Mariano R. Sto. Domingo is an Assistant Research Scientist and Adjunct Faculty at the Department of Psychology of the University of Maryland Baltimore County (UMBC). He received his Ph.D. in Human Services Psychology (Community and Applied Social Psychology Track) from the same university, and studied as an exchange scholar at the Johns Hopkins University. He also earned master’s degrees in social psychology and international relations from the University of the Philippines, and International University of Japan, respectively. Dr. Sto. Domingo has had several years of experience evaluating education-related programs such as the Abell Foundation-funded Baraka School in Kenya, and the Catholic Schools Project that compares Baltimore City’s Catholic schools versus public schools in the city in terms of their impact on minority children’s academic achievement. He was the program coordinator and currently co-investigator of UMBC’s National Institutes of Health funded research grant entitled “Analyzing the Impact of the Meyerhoff Scholarship Program: Pathways to the Ph.D..”

Kenneth I. Maton is Professor of Psychology and director of the Community and Applied Social Psychology Ph.D. Program in Human Services Psychology at University of Maryland Baltimore County (UMBC). He received his Ph.D. degree in Clinical-Community Psychology from the University of Illinois, Champaign-Urbana. His research focuses on minority youth achievement, and includes the longitudinal evaluation of UMBC’s Meyerhoff Scholarship Program. Books include: Investing in Children, Youth, Families and Communities: Strengths-Based Research and Policy (edited volume; American Psychological Association) and Beating the Odds: Raising Academically Successful African American Males (co-author; Oxford University Press). Dr. Maton is past-president of the Society for Community Research and Action (SCRA; APA Division 27), and the 2006 winner of SCRA’s Distinguished Contribution to Theory and Research Award. He serves on the editorial boards of American Journal of Community Psychology and Journal of Community Psychology.