Abstract
The Meyerhoff Scholarship Program, which celebrated its 20th year in 2008, is considered a successful intervention program for increasing the number of underrepresented minorities who earn PhDs or MD/PhDs and pursue research careers in science, technology, engineering, and mathematics (STEM). This article examines the relationship between participation in one specific component of the Meyerhoff Scholarship Program—on-campus, academic year research—and the pursuit of a STEM PhD by thirteen cohorts of program participants. The results indicate that participation in on-campus, academic year research is associated with a substantial increase in the probability of pursuing a STEM PhD. They further suggest that the structure and intensity of the on-campus, academic year research experience matter.
Keywords: Pursuit of a STEM PhD, undergraduate research, minority students
Introduction
The importance of increasing the number of science, technology, engineering and mathematics (STEM) PhDs in the United States has been highlighted in numerous reports over the past decade (Domestic Policy Council, 2006; Jackson, 2004; NSF, 2002; National Science and Technology Council, 2000). A major concern is the threat that the shortage of STEM PhDs poses to American technological global leadership, which is expected to affect the economy of the U.S. and its position as a world leader. An important response to this concern on the part of policy, scientific and educational communities has been to focus on underrepresented minorities as populations of attention because of their chronic underrepresentation in science and engineering on all levels. Compared to their positions in society where Hispanics are 14.8% of the population, African American’s are 13.4%, and Native Americans, including Alaskans, are 1.5%, these groups combined represented less than 10% of STEM doctoral degrees awarded in 2005 (U.S Census, 2007; Hoffer et al., 2005).
Education has been linked as a direct contributor to these efforts because of its ability to increase and maintain a technical workforce (Domestic Policy Council, 2006). However, underrepresented minority students, specifically in STEM disciplines, experience challenges that hinder their success in STEM education, in general, and in completion of STEM doctoral degrees, in particular (Citation Removed; Chubin, May, & Babco, 2005). These challenges include under preparation among some underrepresented minority students as well as discrimination of prepared minority students in the competitive environments of the STEM fields. For example, many reports note that although minority students who enter STEM undergraduate programs are as equally represented and talented as their non-minority counterparts, minority students often do not persist through their undergraduate programs, which directly relates to their opportunity to pursue a STEM PhD (Summers & Hrabowski, 2006).
These problems have been addressed by the creation of numerous STEM education enrichment and/or scholarship programs to increase the number of STEM undergraduate majors and PhDs, especially from underrepresented groups (Maton & Hrabowski, 2004; Summers & Hrabowski, 2006; Barlow & Villarejo, 2004). Since original research is an essential phase in obtaining a PhD as well as maintaining technological innovation, undergraduate research opportunities are often a major element of such programs. Although the most critical research experience of a scientist’s career may be his/her PhD dissertation, many underrepresented minority graduate students do not reach this point in their intended careers. Therefore, a thorough understanding of the influence of undergraduate research experiences on underrepresented minorities is one essential component for advancing the understanding of their successful pursuit of STEM PhDs.
Although the belief that participation in undergraduate research opportunities impacts a student’s pursuit of a STEM PhD is widespread, empirical research on the relationship between such intervention policies and programs and subsequent pursuit of a STEM PhD is scant. The limited literature regarding this relationship focuses primarily on the relationship between summer research programs and pursuit of a STEM PhD rather than the relationship between research experiences that occur on-campus during the academic year and pursuit of a STEM PhD. However, there are several reasons to expect the relationship between participation in on-campus, academic year research and pursuit of a STEM PhD to differ from the relationship between summer research programs and pursuit of a STEM PhD. For one, participating in on-campus, academic year research can continue for longer time periods and give students a more in-depth view of research. The continuous research experience may also lead to the development of culture, relationships, and scientific results that are more similar to the PhD process. As a result of the differences in on-campus, academic year research when compared to other types of research experiences, our research addresses the relationship of this specific form of research experience and subsequent pursuit of a STEM PhD.
Specifically, this study reports new empirical evidence regarding the relationship between participation in on-campus, academic year research by undergraduates in the Meyerhoff Scholarship Program at the University of Maryland, Baltimore County (UMBC) and their subsequent pursuit of a STEM PhD. Three types of on-campus, academic year research experiences are considered: participation in UMBC’s Minority Access to Research Careers Undergraduate Student Training in Academic Research (MARC U-STAR) program; completion of research courses for academic credit; and participation in UMBC’s undergraduate research symposium.
Our study has several strengths that enable it to make an important contribution to understanding the relationship between participation in on-campus, academic year research and subsequent pursuit of a STEM PhD. Our study uses statistical techniques as well as additional variables to control for unobserved factors that may influence Meyerhoff participants conducting research as well as pursuing a STEM PhD. The collection of data and use of variables for participation in on-campus, academic year research only (as opposed to other types of research) also provide a method to distinguish between the various types of research. The sample used in this study is also a majority minority sample, which addresses the issue of beginning to understand the relationship between on-campus, academic year research and subsequent enrollment in STEM PhD programs for underrepresented groups, a distinct but important group of collegians.
It is, however, important to note that that our study has several limitations. First, it is not based on a randomized control trial. Therefore, while we can analyze the relationship between participation in on-campus, academic year research and subsequent enrollment in a STEM PhD program, the study provides very limited support for causal claims. Second, the Meyerhoff participants are a very selective group. While, as noted above, our study sample is a majority minority sample, the average characteristics of this sample are very different than those of underrepresented minority students, in general. Thus, the generalizability of our results is limited.
The next section of this paper summarizes the existing literature regarding the benefits of undergraduate research for graduate and professional school. A brief description of the Meyerhoff Scholarship Program and opportunities for participation in on-campus, academic year research available to Meyerhoff participants is then provided. We then discuss the data used in this study along with the analyses we conducted. Following that, we present the results of our analyses. The final section of this paper contains a discussion and conclusions.
Undergraduate Research and PhD Outcomes
The present culture of undergraduate research found across the nation is thought to have originated in 1969 at the Massachusetts Institute of Technology. The university’s administration declared that undergraduate students should be given the advantages of mentor-protégée relationships, allowing them to be recognized as an important part of the scholarly community. MIT was followed by other research universities, the National Science Foundation, as well as other professional and private scientific organizations, such as the Council on Undergraduate Research, that have worked together to build the current culture of undergraduate research. This culture is built on the creation of organizations, policies and programs that provide funding to support undergraduate research (Merkel, 2001). However, this focus on research has also been met with opposition from education reform and teaching organizations that deemed research universities were neglecting teaching for research (Boyer Commission, 1998). As a result of studies of major research universities in the US, the Commission recommended inquiry-based learning found in research experiences be extended to all students. These efforts to support undergraduate research have created numerous opportunities for undergraduate students to participate in research programs. Unfortunately, analysis of the relationship between undergraduate research opportunities and PhD outcomes is limited.
The earliest studies on the results of undergraduate research programs addressed general benefits provided by undergraduate research, such as increasing student success in completing bachelor’s degrees. One of the earliest of these studies analyzed the University of Michigan’s Undergraduate Research Opportunity Program (UROP). Using university registrar data for randomly assigned experimental (those accepted in UROP) and control (not accepted into UROP) groups, Nagda and colleagues (1998) found that participation in UROP increased retention rates for African-American students performing below the median for their ethnic group. Additionally, they found positive trends in retention for Hispanic and White students who participated in UROP during their sophomore year. The study suggested that amplified student-faculty interaction through research partnerships enhanced student success as a result of improved academic, social and institutional integration of UROP participants. In their study of participants in the Biology Undergraduate Scholars Program (BUSP) at the University of California, Davis, Barlow and Villarejo (2004) report impressive evidence regarding the relationship between research participation and student persistence as well as performance. In particular, they found that relative to BUSP students who did not participate in undergraduate research, those BUSP students who participated in undergraduate research had significantly higher probabilities of graduating from the University of California, Davis in any major, graduating with a major in the biological sciences, and graduating in biology with a cumulative GPA above 3.0.
As investigations relating students’ undergraduate research experiences with increased success expanded, hypotheses and studies were developed to test whether these experiences would better prepare them for graduate school and research careers. In a subsequent study of UROP that employed a randomized control design, Hathaway and colleagues (2002) concluded that participation in undergraduate research increased the probability of pursuing graduate education and post-undergraduate research activity. Similarly, Russell and colleagues’ (2007) investigation of nationwide undergraduate research opportunities found that 30% of undergraduates with more than 12 months of research experience expected to obtain a PhD compared to only 13% and 8% of students with less than three months of research experience and no research, respectively.
Recent efforts to better understand the relationship between undergraduate research and pursuit of a STEM PhD have led to studies that address the personal, professional and cognitive benefits of undergraduate research. Lopatto’s (2003) initial studies were aimed at providing a more precise assessment of benefits from summer undergraduate research by compiling essential features recommended by faculty research mentors with those presented by students. The findings revealed that faculty and students agreed on benefits associated with career paths, in-depth learning, research and laboratory skills and developing relationships. However, the two groups disagreed on the importance of tasks such as reading relevant literature. Subsequent studies by Lopatto also found that extended research experiences were thought to lead to higher rates of STEM PhD’s by increasing research efficacy, PhD anticipation and science career decisions of undergraduate students already in the PhD pipeline (2007).
In addition to reporting that undergraduate research participants are more likely to expect to obtain a PhD, Russell and colleagues (2007) provide insight into several outcomes of undergraduate research opportunities in their study. For example, participants reported increases in understanding how to conduct a research project, confidence in research skills, and awareness of the environment and processes of graduate school. At four liberal arts institutions, Hunter and colleagues (2006) interviewed STEM students who conducted summer research along with their faculty advisors and students who had not conducted undergraduate research. Their findings suggest that undergraduate research supports PhD and research career success as a result of professional, personal and cognitive gains towards “becoming a scientist” that collegians acquire. Likewise, Bauer and Bennett (2003) interviewed alumni of the University of Delaware who had conducted undergraduate research and alumni who had not done so. Bauer and Bennett’s primarily qualitative analysis discovered increased pursuit of graduate studies by participants in undergraduate research. The study also reported increased skills that were grouped into: science, math, logic and problem solving; literature, language and mastery of contexts; and personal initiative and communication.
Although previous research has provided important findings about the relationship between undergraduate research and expectations of and pursuit of PhDs as well as some initial understanding of how undergraduate research produces these results, the general body of research suffers from several limitations. Several of the studies (e.g., Hunter et al., 2007; Bauer and Bennett, 2003) are qualitative in nature. Although these studies provide critical information on the mechanisms through which research is effective, they do not illustrate direct empirical relationships between undergraduate research and PhD entry or completion. In addition, both of these studies reported ethnic minorities as less than 5% of their sample. Therefore, a direct application of their results to other programs or institutions with more diverse populations may not be possible. Since many of the undergraduate research opportunities target underrepresented groups of students in an effort to increase the number of minority scientists and PhDs, both qualitative and quantitative studies that focus on minority populations are needed. Thus, similar studies on the benefits for underrepresented minorities are an essential future step in expanding this area of research.
In addition, the quantitative studies that have been conducted leave some doubt regarding whether the estimated effects are in fact caused by participation in undergraduate research. For example, it is unclear whether the differences in the degree expectations among those with different levels of undergraduate research experience reported by Russell and colleagues (2007) represent the effect of undergraduate research experience on degree expectations or the effect of degree expectations on undergraduate research participation. In addition, although Hathaway and colleagues (2002) employ a randomized control group design, their analysis compares endogenously formed subgroups. Specifically, they compare those who reported participating in undergraduate research, i.e., those assigned to the UROP group plus crossovers (students not assigned to the UROP group who nonetheless participate in undergraduate research) with those who did not report participating in undergraduate research (i.e., those assigned to the control group who were not crossovers).
An additional limitation of the body of research is not distinguishing between the types of undergraduate research experiences in which students participate. As the culture of undergraduate research has evolved so have the types of research opportunities that are made available to students. For example, students can participate in summer or academic year research experiences at their home university, other nearby universities, government agencies or private organizations. With the various classifications of research opportunities, one can hypothesize about differences in the benefits gained from each type of experience. However, since the previous studies have not distinguished between the type or timing of undergraduate research experiences of their informants, it is impossible to determine the relationship between each type of research and PhD outcomes. Therefore, our focus on the increase in STEM PhD enrollment associated with participation in on-campus, academic year research separate from other types of undergraduate research participation is motivated by the importance of understanding the influence of each type of research separately as well as the lack of research of this detail that is currently available.
On-campus, academic year research is defined in this study as research experiences by undergraduates that occur on the students’ native campus during the academic school year only. These experiences occur while an undergraduate student is also enrolled in courses for STEM bachelor’s degree completion and possibly being involved in other activities on campus. These students are conducting research with pressures similar to those in graduate school, where students often serve as teaching or graduate assistants, take classes and have research responsibilities. Participation in on-campus, academic year research also often involves attending research group meetings and developing in-depth relationships with faculty mentors and graduate students similar to the kind of relationships developed in graduate school. Since participation in on-campus, academic year research provides several similarities to graduate study, we anticipate that these students will have even more insight into the environment and processes of graduate school. These insights may increase their graduate school decisions and pursuit of a STEM PhD differently and lead to more influence than other types of undergraduate research experiences. Therefore, we believe that participation in on-campus, academic year research differs from other types of undergraduate research. Our beliefs are supported by findings of Russell and colleagues for undergraduates with 12 months or more of undergraduate research experience and Hathaway and colleagues’ assessment of the UROP program that enrolls students in research during the academic year (Russell et al., 2007; Hathaway et al., 2002).
The Meyerhoff Scholarship Program
The primary goal of the Meyerhoff Scholarship Program is to increase the number of underrepresented minorities who earn PhDs or MD/PhDs and pursue research careers in STEM fields. The program, which celebrated its twentieth year in 2008, currently enrolls 215 scholars and has successfully graduated more than 550 STEM majors overall. The program is situated on the campus of a doctoral, research-intensive university with a diverse student population (34% minority) of approximately 9,700 undergraduate and 2,600 graduate students (UMBC Facts, 2008). More than half of the undergraduates and 60 percent of the doctoral students at UMBC are pursuing STEM degrees (UMBC Office of Institutional Advancement, 2007).
The Meyerhoff Scholarship Program is a comprehensive, strengths-based approach to intervention and research that incorporates fourteen different program components geared toward supporting undergraduate, PhD or MD/PhD, and research career success for each scholar. Of particular relevance to the current study is the program’s multifaceted focus on research with several objectives, such as exposing students to research as early as possible in their academic careers. The program requires that students participate in multiple summer research internships during the summers in which they are scholars (Maton & Hrabowski, 2004). There are several aims for these summer research experiences, which include but are not limited to familiarizing scholars with the practice of STEM research and careers and strengthening a student’s resume (Gordon & Bridglall, 2004).
Various mechanisms provide Meyerhoff participants with opportunities to participate in on-campus, academic year research at UMBC. These mechanisms include research courses in academic departments, an undergraduate research symposium, research programs for specific groups of students, as well as paid or voluntary research opportunities by students in each academic department. Our study focused on the first three types of research experiences mentioned above.
Research opportunities at UMBC are offered to all undergraduate students as a part of their professional development (UMBC Office of Undergraduate Education, 2008). Research courses are offered by each department as upper-level courses to both honors and non-honors collegians. The courses require permission from faculty and being accepted into the departmental honors program if the course is an honors course. Students are also encouraged to present their research in UMBC’s annual research symposium for undergraduates, which is entitled Undergraduate Research and Creative Achievement Day. The research symposium for undergraduates has been held annually since the spring of 1997.
The research program for specific students is the Minority Access to Research Careers Undergraduate Student Training in Academic Research (MARC U-STAR) program, which requires its full scholarship recipients to conduct research during the academic year. For many Meyerhoff scholars, MARC U-STAR funding is a substitute for Meyerhoff Scholarship Program funding when the students, who must be junior and senior level honors students, have established research career interests. UMBC’s MARC U-STAR program, which began in 1997, is one of a number of similar NIH-funded undergraduate programs throughout the US. The award requirements of the MARC U-STAR grant for each program include providing summer and academic year research experiences for undergraduates (MARC Program Description, 2008).
As discussed above, few studies examine the relationship between undergraduate research, in general, and pursuit of a STEM PhD and even fewer consider the relationship between participation in on-campus, academic year research, specifically, and pursuit of a STEM PhD. Although the Meyerhoff Scholarship Program has been widely studied, the previous studies have not addressed the relationship between the research components of the program and pursuit of a STEM PhD. The limited information available on the relationship between participation in on-campus, academic year research and the subsequent pursuit of a STEM PhD has led us to the following research questions:
What is the relationship between participation in on-campus, academic year research and subsequent pursuit of a STEM PhD?
Is this relationship uniform for different groups of intended majors?
Is this relationship uniform for different types of on-campus research experiences?
Data and Methods
Study Sample
Our intended study sample consists of all students in the first thirteen cohorts entering the Meyerhoff Scholarship Program, that is, Meyerhoff participants entering UMBC from 1989 to 2001. Meyerhoff participants who entered UMBC in 2002 or later were excluded primarily because too many were still enrolled as undergraduates in a STEM major. There are a total of 517 students in these included cohorts. Of these 517, it was necessary to drop 47 cases that were missing PhD status. It was also necessary to drop an additional 25 cases that were missing data on parental education. Finally, because of their extremely low representation in our sample (n=4), Hispanic participants were dropped from our sample. Hence, our analytic sample consisted of 441 non-Hispanic Meyerhoff participants.
Variable Definitions and Sources
Our dependent variable is whether the individual was enrolled in, or had completed, a STEM PhD or MD/PhD program as of June 2008. For convenience, we refer to that outcome as pursuit of a STEM PhD. Students who applied to, but did not enroll in, a STEM PhD or MD/PhD program or had dropped out of such a program were not considered to be pursuing a STEM PhD. These data are acquired annually from graduate transcripts and verbal reports obtained from registrars’ offices at institutions where Meyerhoff participants enrolled. Of the 441 members of our analytic sample, 31% are currently pursuing or have completed a STEM PhD or MD/PhD program.
Our key independent variable is undergraduate participation in on-campus, academic year research. Three specific types of on-campus, academic year research activities are considered:
MARC U-STAR participation;
Research course credits; and
Participation in UMBC’s annual undergraduate research symposium.
Data regarding MARC U-STAR participation was collected from the MARC U-STAR and Meyerhoff Scholarship Program operational data. Research courses were identified through review of the UMBC undergraduate course catalog descriptions for courses that required laboratory based research, publishing a scholarly paper, or other common scientific research aspects. By using this process many independent study courses were not considered research courses for this study since their course descriptions led us to conclude that they did not meet the requirements for on-campus, academic year research. Data regarding research courses taken by each Meyerhoff participant were obtained from the UMBC registrar. As part of the celebration of the tenth year anniversary of UMBC’S annual undergraduate research symposium in Spring 2007, a cumulative list of student, mentor and research participation information was compiled by the undergraduate research staff within UMBC’s Office of Undergraduate Education.. This cumulative list was used as a source for the data set to identify Meyerhoff participants who presented the results of their research experiences as participants in this symposium.
Due to variation in their length, intensity and structure, it is not clear that each of these three types of experience offer the same benefits to participants. For example, although the annual undergraduate research symposium requests information on a student’s UMBC mentor and states that the presentations are usually from students who completed research on campus during the academic year, these are not mandatory requirements (UMBC Office of Undergraduate Education, 2008). Therefore, a student could present research conducted during a summer program or on a different campus. Moreover, the symposium’s goal of exposing students to research for professional development while at UMBC results in a wide range of levels of research presentations from literature reviews to research projects that have been peer-reviewed for publication. Therefore, it is not clear that this type of experience offers the same level of intensity and structure as MARC U-STAR participation or completion of research courses for credit. Consequently, it is quite plausible that the increase in the probability of pursuing a STEM PhD associated with participation in UMBC’s annual undergraduate research symposium will be smaller than the increases associated with participation in other types of on-campus, academic year research. Similarly, completing two or more research courses for credit is likely to provide students with a more in-depth view of research as well as greater interaction with a faculty mentor and other researchers than completing only one research course for credit.
To address this issue, two different definitions of participation in on-campus, academic year research were used in our analysis. Under the first definition, a Meyerhoff participant is considered to have participated in on-campus, academic year research if he or she has participated in MARC U-STAR, completed at least one research course for credit or participated in UMBC’S annual undergraduate research symposium. Under the second definition, which we refer to as “intensive” forms of on-campus, academic year research, a Meyerhoff participant is considered to have participated in on-campus, academic year research if he or she has participated in MARC U-STAR or completed at least two research courses for credit. Of the 146 members of our sample who were considered to have participated in on-campus, academic year research under the first definition, 107 were also considered to have participated under the second.
Finally, our analyses include a variety of control variables. These include variables pertaining to participants’ demographic characteristics, pre-college educational background, parental educational attainment, summer research experience as an undergraduate, and major declared at the beginning of the freshman year. Data regarding some of these control variables were obtained from Meyerhoff Scholarship Program records. Data regarding other control variables were obtained from questionnaires that participants completed during the “summer bridge” component of the Meyerhoff Scholarship Program in which all scholars participate prior to their freshman year. Table 1 identifies which of these sources was used for each control variable.
TABLE 1.
DESCRIPTIVE STATISTICS OF INDEPENDENT VARIABLES (N=441)
| VARIABLE | SOURCE OF DATA* | CATEGORY | MEAN |
|---|---|---|---|
| Participation in On- Campus Academic Year Research | UMBC Registrar, UMBC OUE, MMOD | One Research Course | .100 |
| Two or More Research Courses | .095 | ||
| UMBC’s Annual Undergraduate Research Symposium | .107 | ||
| MARC U-STAR | .206 | ||
|
| |||
| One Research Course, not MARC U- STAR | .052 | ||
| Two or More Research Courses, not MARC U-STAR | .036 | ||
| UMBC’s Annual Undergraduate Research Symposium Only | .036 | ||
| MARC U-STAR | .206 | ||
|
| |||
| Intensive Form of On-Campus Academic Year Research** | .243 | ||
|
| |||
| Any Form of On-Campus Academic Year Research*** | .331 | ||
|
| |||
| Participation in Summer Research | MMOD | Following:Freshman Year | .435 |
| Following:Sophomore Year | .596 | ||
| Following:Junior Year | .608 | ||
| Following:Senior Year | .181 | ||
|
| |||
| Gender | MMOD | Male | .494 |
|
| |||
| Ethnicity**** | MMOD | African-American | .841 |
| Asian/Pacific Islander | .077 | ||
| Caucasian | .082 | ||
|
| |||
| Intended Major | MMOD | Other & Undeclared | .218 |
| Engineering & Computer Science | .297 | ||
| Natural Sciences | .485 | ||
|
| |||
| Parental Education | MMOD | Less than an Associate’s Degree | .211 |
| Assoc./Bachelors | .279 | ||
| Graduate/Professional | .510 | ||
|
| |||
| Math SAT Score | MMOD | 654 | |
|
| |||
| High-School GPA | MMOD | 3.72 | |
UMBC OUE = UMBC Office of Undergraduate Education; MMOD – MARC U-STAR/Meyerhoff Scholarship Program Operational Data.
Participated in MARC U-STAR or completed at least two research courses for credit.
Participated in UMBC’s annual undergraduate research symposium or completed one research for credit and did not partcipate in MARC U-STAR.
Although Hispanic is a possible category for ethnicity, the analysis did not include this group due to its small size (4 students) within the sample.
Demographic characteristics that were used as control variables were gender and ethnicity. Ethnicity was categorized as African-American, Asian/Pacific Islander or Caucasian. Pre-college educational background variables considered were high school GPA and math SAT score. Parental education was operationally defined as the highest degree obtained by either parent of the Meyerhoff participant. Three categories of parental education were considered in our analytical models: no college degree; associate or bachelor’s degree; and at least some post-baccalaureate study. Majors declared at the beginning of the freshman year were aggregated into three categories: natural and biological sciences; engineering and computer science; and other or undeclared.
Table 1 contains descriptive statistics of the independent and control variables for our sample. Approximately one-third of Meyerhoff participants participated in some form of on-campus, academic year research, while nearly one-quarter participated in an “intensive” form of on-campus, academic year research. However, as Figure 1 shows, there are considerable differences among the cohorts in terms of participation in on-campus, academic year research, particularly between the first five cohorts and the next eight cohorts in terms. To a large extent, this is due to the start of the MARC U-STAR program and UMBC’S annual undergraduate research symposium in the fall of 1997. Since the MARC U-STAR program enrolls juniors and seniors, the first Meyerhoff Scholarship Program cohort that would be expected to be able to enroll in the MARC U-STAR program is cohort 6 (i.e., those whose college careers began in the fall of 1994). Indeed, as illustrated in Figure 1, no members of the first five cohorts participated in MARC U-STAR, while 11% of cohort 6 and between 26% and 38% of subsequent cohorts did so. Similarly, only one member of the first five cohorts participated in UMBC’s annual undergraduate research symposium, as compared to 5% of the members of cohorts 6 through 13. The rate at which members of the various cohorts completed research courses for credit varies considerably less among the cohorts.
FIGURE 1.
RATES OF PARTICIPATION IN VARIOUS FORMS OF ON-CAMPUS, ACADEMIC YEAR RESEARCH BY COHORT
Nearly 80% of the participants declared a STEM major their freshman year. The mean high school GPA and SAT math score for sample members were 3.72 and 654, respectively. The sample is approximately 49% male, 84% African-American, 8% Asian/Pacific Islander, and 8% Caucasian. In terms of parental educational background, only 21% of the students’ parents had not earned at least an associate’s or bachelor’s degree, while 51% had at least some graduate or professional study.
Data Analysis
Our main data analysis consists of a series of probit regression models. As noted above, in each of these models the dependent variable is pursuit of a STEM PhD, i.e., whether or not the Meyerhoff participant is enrolled in or has completed a STEM PhD. The key independent variable in our model is participation in on-campus, academic year research. As discussed above, participation in on-campus, academic year research is defined in two ways. The key distinction between these two definitions is how the 39 Meyerhoff participants whose only form of participation in on-campus, academic year research is UMBC’s annual undergraduate research symposium or completion of one research course for credit are categorized. Our models also contain a series of control variables, including math SAT score, high-school GPA, ethnicity, gender, parental education, intended major, and participation in summer research following the student’s freshman, sophomore, junior, and senior years of college. Each model was estimated for all non-Hispanic Meyerhoff participants as well as for African-American Meyerhoff participants only. (The number of Caucasian and Asian/Pacific Islander Meyerhoff participants was insufficient to estimate the models for such participants separately). In the latter case, of course, ethnicity was not included as a control variable in the model.
A potential concern associated with our probit regression models is that the estimates of the coefficient of participation in on-campus, academic year research may be biased due to the presence of unmeasured variables that affect both the probability of participation in on-campus, academic year research and the probability of pursuing a STEM PhD. To address this concern, we considered an instrumental variable estimator. Figure 1 suggests that an ideal excluded instrument for participation in on-campus, academic year research is whether the participant was in cohorts 1–5 or in cohorts 6–13. As noted above, members of the later cohorts had access to two forms of on-campus, academic year research to which members of the earlier cohorts did not, namely MARC U-STAR and UMBC’s annual undergraduate research symposium. Use of this instrument is somewhat analogous to using random assignment as an instrument in analyses of randomized control trials involving “crossovers” (see Angrist et al., 1996; Gennetian et al., 2005). Assuming that there are no “defiers”, the group of members of the earlier cohorts who participated in on-campus, academic year research are composed solely of “always-takers”, while the group of members of earlier cohorts who did not participate in on-campus, academic year research are composed of “compliers” and “never-takers”. In contrast, the group of members of the later cohorts who participated in on-campus, academic year research are composed of “compliers” and “always-takers”, while the group of members of later cohorts who did not participate in on-campus, academic year research are composed solely of “never-takers”. The instrumental variable (IV) estimates produced in this situation can be interpreted as local average treatment effects for “compliers”. These effects represent the change in the probability of entry into STEM PhD programs resulting from the increase in participation in on-campus, academic year research that is induced by the additional opportunities for on-campus, academic research that were available to the later cohorts.
Hence, we reestimated our basic model using this excluded instrument. Two requirements for excluded instruments are essential (see, for example, Ludwig and Bassi, 1999; Miller et al., 2008; and Wooldridge, 2009). The first is that they be relevant, i.e., strong predictors of the first-stage dependent variable (in our case participation in on-campus, academic year research). To test this condition, we consider the F-statistic associated with the excluded instrument (whether the participant was a member of cohorts 6–13) in the first-stage equation in our IV estimation. The second essential requirement for excluded instruments is that the exclusion restriction must be valid, that is the excluded instrument should not be a predictor of the outcome variable—pursuit of a STEM PhD—except through its relationship with participation in on-campus, academic year research. To test this condition, we estimated a reduced form equation, that is, a probit model in which the dependent variable is pursuit of a STEM PhD and the independent variables include whether or not the Meyerhoff participant was in cohorts 6–13 (the excluded instrument) and all control variables included in our main probit model, but not participation in on-campus, academic year research. We then tested the null hypothesis that in this reduced form equation the coefficient of the indicator variable for whether the Meyerhoff participant was in cohorts 6–13 is zero. Rejecting this null hypothesis constitutes evidence that the exclusion restriction is not valid.
We also conducted Wu-Hausman tests to ascertain whether participation in on-campus, academic year research is, in fact, endogenous. This test consisted of adding the residual from the first-stage model to the probit model for pursuit of a STEM PhD and testing the significance of the coefficient of that residual. Rejection of the null hypothesis that this coefficient is zero constitutes evidence that participation in on-campus, academic year research is endogenous, which indicates that the IV estimates should be used rather than the non-instrumented estimates.
To test whether the coefficient of participation in on-campus, academic year research was uniform among Meyerhoff participants in the three groups of majors we added a pair of product terms (participation in on-campus, academic year research times engineering major and participation in on-campus, academic year research times science major) to our basic model. In this model, the coefficient of participation in on-campus, academic year research represents the increase in the probit function of the probability of pursuing a STEM PhD associated with participation in on-campus, academic year research for Meyerhoff participants whose intended major was other or undeclared. The coefficients of the two product terms represent the differences between:
the increase in the probit function of the probability of pursuing a STEM PhD associated with participation in on-campus, academic year research for Meyerhoff participants whose intended major was engineering and computer science and the corresponding increase for Meyerhoff participants whose intended major was other or undeclared; and
the increase in the probit function of the probability of pursuing a STEM PhD associated with participation in on-campus, academic year research for Meyerhoff participants whose intended major was one of the natural sciences and the corresponding increase for Meyerhoff participants whose intended major was other or undeclared.1
We then tested the null hypothesis that the coefficients of both of these two product terms are zero. Rejection of this null hypothesis indicates that the increases in the probability of pursuing a STEM PhD associated with participation in on-campus, academic year research are not uniform among the three groups of majors.
To test whether the relationship between participation in on-campus, academic year research and subsequent pursuit of a STEM PhD is uniform for different forms of on-campus, academic year research, we estimated a probit model that includes a series of indicator variables representing participation in four different types of on-campus, academic year research, rather than a single indicator variable for participation in on-campus, academic year research. The same set of control variables that were used in the basic model are used in this model. We then tested the null hypothesis that the coefficients of participation in these different types of on-campus, academic year research are equal to each other. Rejection of this null hypothesis constitutes evidence that the increases in the probability of pursuing a STEM PhD associated with participation in different types of on-campus, academic year research are heterogeneous. We also estimated a probit model that includes a pair of indicator variables representing type of on-campus, academic year research in which the student participated. The first of these indicator variables represents participation in UMBC’S annual undergraduate research symposium or completion of one research course for credit (but not MARC U-STAR participation); the second represents completion of two or more research courses for credit or MARC U-STAR participation. Thus, this model assumes that the coefficient of participation in UMBC’s annual undergraduate research symposium only is equal to the coefficient of completing one research course for credit and that the coefficient of completing two or more research courses for credit is equal to the coefficient of MARC U-STAR participation. To test this assumption, we applied a likelihood ratio test to these two models. We also tested the null hypothesis that the coefficients of participation in the two types of on-campus, academic year research contained in the more parsimonious model are equal to each other.
Results
Table 2 contains the results of our probit regression models. For both estimation samples and for both definitions of on-campus, academic year research, the coefficients of participation in on-campus, academic year research are positive and statistically significant. Based on these estimates, the estimated probability of an “average” non-Hispanic Meyerhoff participant who participates in any form of on-campus, academic year research pursuing a STEM PhD is 17.1 percentage points greater than the estimated probability of an “average” non-Hispanic Meyerhoff participant who does not participate in on-campus, academic year research pursuing a STEM PhD.2 The estimated increase in the probability of pursuit of a STEM PhD associated with participation in an “intensive” form of on-campus, academic year research is 27.1 percentage points. For an “average” African-American Meyerhoff participant the estimated increases in the probability of pursuit of a STEM PhD associated with participation in any form of on-campus, academic year research and participation in an “intensive” form of on-campus, academic year research are 21.0 percentage points and 29.0 percentage points, respectively.
TABLE 2.
RESULTS OF ESTIMATION OF PROBIT MODEL OF DETERMINANTS OF PURSUIT OF A STEM PH.D. (ROBUST STANDARD ERRORS IN PARENTHESES)
| ESTIMATION SAMPLE
|
||||
|---|---|---|---|---|
| All Non-Hispanic Meyerhoff Participants | All African-American Meyerhoff Participants | |||
| DEFINITION OF ON-CAMPUS, ACADEMIC YEAR RESEARCH
| ||||
| Any Form | “Intensive” Only | Any Form | “Intensive” Only | |
| Participation in On-Campus, Academic Year Research | .492*** (.152) | .751*** (.172) | .607*** (.166) | .811*** (.187) |
| Participation in Research in Summer Following Freshman Year | .127 (.152) | .0384 (.157) | .233 (.167) | .128 (.173) |
| Participation in Research in Summer Following Sophomore Year | .229 (.156) | .221 (.157) | .197 (.168) | .192 (.169) |
| Participation in Research in Summer Following Junior Year | .377** (.156) | .346** (.158) | .165 (.173) | .134 (.175) |
| Participation in Research in Summer Following Senior Year | .635*** (.174) | .642*** (.176) | .685*** (.191) | .69*** (.192) |
| Intended Major (Reference Category is Other or Undeclared) | ||||
| Engineering or Computer Science | .220 (.207) | .214 (.208) | .130 (.227) | .137 (.227) |
| Natural Sciences | .416** (.198) | .407** (.198) | .443** (.215) | .440** (.215) |
| Math SAT Score | .00288** (.00139) | .00283** (.00140) | .00276* (.00155) | .00278* (.00157) |
| High-School GPA | .0824 (.219) | −.00379 (.223) | .201 (.239) | .114 (.242) |
| Ethnicity (Reference Category is African-American) | ||||
| Asian/Pacific Islander | −.525* (.294) | −.389 (.297) | ||
| Caucasian | .399 (.246) | .512** (.250) | ||
| Female | .143 (.140) | .143 (.141) | .104 (.154) | .096 (.155) |
| Parental Education (Reference Category is No College Degree) | ||||
| AA/BA | −.101 (.206) | −.0979 (.208) | −.179 (.229) | −.168 (.229) |
| At Least Some Graduate or Professional Education | .0916 (.181) | .147 (.183) | .0526 (.197) | .106 (.197) |
| Intercept | −3.83*** (1.19) | −3.48*** (1.2) | −4.06*** (1.35) | −3.72*** (1.37) |
|
| ||||
| n | 441 | 441 | 371 | 371 |
| Log-likelihood | −231.83 | −227.33 | −191.49 | −188.57 |
| Pseudo-r | .15 | .17 | .16 | .17 |
| χ2 | 81.25 | 90.24 | 71.47 | 77.30 |
p<.10;
p<.05;
p<.01
Table 3 summarizes the results of our IV estimation. As the top panel of Table 3 shows, the estimated coefficients of participation in on-campus, academic year research are larger in magnitude than the corresponding non-instrumented estimates contained in Table 2. However, the standard errors of these coefficient estimates are approximately five times larger than their non-instrumented counterparts. As a result, although the non-IV coefficient estimates of participation in on-campus, academic year research are all statistically significant at the .01 level, the IV coefficient estimates of participation in on-campus, academic year research are not statistically significant. The estimated increases in the probability of pursuit of a STEM PhD associated with participation in on-campus, academic year research based on these estimates are also larger than those that are obtained from the non-instrumented analyses. In particular, the estimated increase in the probability of pursuit of a STEM PhD associated with participation in any form of on-campus, academic year research based on the IV coefficient estimates are 27.0 percentage points and 30.9 percentage points for an “average” non-Hispanic Meyerhoff participant and an “average” African-American Meyerhoff participant, respectively. The estimated increase in the probability of pursuit of a STEM PhD associated with participation in an “intensive” form of on-campus, academic year research based on the IV coefficient estimates are 29.4 percentage points and 37.8 percentage points for an “average” non-Hispanic Meyerhoff participant and an “average” African-American Meyerhoff participant, respectively.
TABLE 3.
RESULTS OF IV ESTIMATION OF PROBIT MODEL OF DETERMINANTS OF PURSUIT OF A STEM PH.D. (ROBUST STANDARD ERRORS IN PARENTHESES)
| ESTIMATION SAMPLE
|
||||
|---|---|---|---|---|
| All Non-Hispanic Meyerhoff Participants | All African-American Meyerhoff Participants | |||
| DEFINITION OF ON-CAMPUS, ACADEMIC YEAR RESEARCH
| ||||
| Any Form | “Intensive” Only | Any Form | “Intensive” Only | |
| Participation in On-Campus, Academic Year Research | .766 (.774) | .812 (.889) | .883 (.819) | 1.05 (.977) |
| Participation in Research in Summer Following Freshman Year | .0813 (.208) | .0263 (.244) | .194 (.217) | .0763 (.292) |
| Participation in Research in Summer Following Sophomore Year | .202 (.171) | .216 (.173) | .170 (.186) | .170 (.191) |
| Participation in Research in Summer Following Junior Year | .352** (.167) | .341** (.168) | .138 (.182) | .110 (.186) |
| Participation in Research in Summer Following Senior Year | .602*** (.205) | .638*** (.189) | .648*** (.229) | .667*** (.221) |
| Intended Major (Reference Category is Other or Undeclared) | ||||
| Engineering or Computer Science | .213 (.205) | .214 (.206) | .118 (.222) | .135 (.221) |
| Natural Sciences | .350 (.273) | .396 (.251) | .378 (.288) | .397 (.278) |
| Math SAT Score | .00290* (.00163) | .00282* (.00163) | .00271 (.00179) | .00272 (.00181) |
| High-School GPA | .0248 (.272) | −.0179 (.293) | .144 (.282) | .0568 (.315) |
| Ethnicity (Reference Category is African-American) | ||||
| Asian/Pacific Islander | −.468 (.343) | −.371 (.390) | ||
| Caucasian | .413 (.256) | .522* (.285) | ||
| Female | .133 (.145) | .142 (.145) | .0832 (.167) | .0811 (.168) |
| Parental Education (Reference Category is No College Degree) | ||||
| AA/BA | −.119 (.205) | −.100 (.208) | −.209 (.241) | −.183 (.235) |
| At Least Some Graduate or Professional Education | .0846 (.184) | .150 (.192) | .0322 (.207) | .107 (.197) |
| Intercept | −3.62*** (1.37) | −3.42** (1.44) | −3.79** (1.60) | −3.45* (1.76) |
|
| ||||
| F-Statistic for Significance of Excluded Instrument in First- Stage Equation | 15.17 (p=.0001) | 18.26 (p<.0001) | 13.49 (p=.0003) | 12.78 (p=.0004) |
| χ2 Statistic for Significance of Excluded Instrument in Reduced Form Equation | 1.02 (p=.313) | same as column 1 | 1.24 (p=.265) | same as column 3 |
| χ2 Statistic for Significance of First- Stage Residual in Probit Model Predicting Pursuit of a STEM PhD (Wu-Hausman Test) | 0.11 (p=.741) | 0.00 (p=.948) | 0.10 (p=.754) | 0.05 (p=.824) |
|
| ||||
| n | 441 | 441 | 371 | 371 |
| Log-likelihood | −462.98 | −401.33 | −385.9 | −337.78 |
| χ2 | 65.86 | 62.78 | 58.28 | 55.71 |
p<.10;
p<.05;
p<.01
Our results also suggest that the variable indicating whether the Meyerhoff participant was in cohorts 6–13 performs well as an excluded instrument. The first row of the middle panel of Table 3 indicates that weak instruments are not likely to be a problem in this case. For both estimation samples and for both definitions of on-campus, academic year research, the variable indicating whether the student was in cohorts 6–13 is a strong predictor of participation in on-campus, academic year research. In addition, as the second row of the middle panel of Table 3 indicates, the exclusion restriction appears to be valid. That is, whether the student was in cohorts 6–13 is not a predictor of pursuit of a STEM PhD in the reduced form equation for either of the estimation samples.3
The third row of the middle panel of Table 3 summarizes the results of the Wu-Hausman F-tests we conducted. These tests indicate that there is no reason to reject the null hypothesis that participation in on-campus, academic year research is exogenous. Thus, it appears that the coefficient estimates of participation in on-campus, academic year research contained in Table 2 are unbiased. Given that the standard errors of the coefficient estimates produced by the IV estimation procedure are approximately five times larger than their non-instrumented counterparts and the failure based on the Wu-Hausman tests to reject the null hypothesis that participation in on-campus, academic year research is exogenous, we believe the non-instrumented estimates are preferable. In any case, they are more conservative than the IV estimates.
Table 4 contains the results of our analyses to test whether the relationship between participation in on-campus, academic year research and subsequent pursuit of a STEM PhD are uniform for different groups of intended majors. The first panel shows coefficient estimates of the product terms for on-campus, academic year research participation for engineering/computer science and natural science majors, respectively, that were added to our base probit model. The second panel contains estimates of the increases in the probability of pursuing a STEM PhD associated with on-campus, academic year participation for different groups of intended majors.4
TABLE 4.
SELECTED RESULTS OF PROBIT ESTIMATION OF DETERMINANTS OF PURSUIT OF A STEM PH.D. REFLECTING HETEROGENEITY OF RELATIONSHIPS AMONG TYPES OF MAJOR
| ESTIMATION SAMPLE
|
||||
|---|---|---|---|---|
| All Non-Hispanic Meyerhoff Participants | All African-American Meyerhoff Participants | |||
| DEFINITION OF ON-CAMPUS, ACADEMIC YEAR RESEARCH
| ||||
| Any Form | “Intensive” Only | Any Form | “Intensive” Only | |
| Coefficient Estimates
|
||||
| Participation in On-Campus, Academic Year Research | .676* (.396) | 1.09** (.470) | .911** (.424) | 1.37*** (.509) |
| Participation in On-Campus, Academic Year Research X Engineering/Computer Science Major | −.0831 (.486) | −.0903 (.563) | −.0702 (.524) | −.104 (.607) |
| Participation in On-Campus, Academic Year Research X Natural Sciences Major | −.267 (.435) | −.486 (.500) | −.479 (.468) | −.827 (.540) |
|
| ||||
| Estimated Marginal Increases in the Probability of Pursuing a STEM PhD Associated with Participation in On- Campus, Academic Year Research for:
|
||||
| Other/Undeclared Major | .205 | .362 | .282 | .459 |
| Engineering/Computer Science Major | .203 | .359 | .276 | .438 |
| Natural Sciences Major | .152 | .230 | .163 | .208 |
|
| ||||
| n | 441 | 441 | 371 | 371 |
| Log-likelihood | −231.56 | −226.49 | −190.58 | −186.29 |
| χ2 Relative to Base Model | 0.54 | 1.69 | 1.81 | 4.56 |
| p-value of χ2 | 0.764 | 0.430 | 0.404 | 0.102 |
| Pseudo r2 | 0.15 | 0.17 | 0.16 | 0.18 |
p<.10;
p<.05;
p<.01
Regardless of the estimation sample and which definition of on-campus, academic year research is used, our analysis gives no reason to reject the null hypothesis that both product terms are zero. That is, it is reasonable to conclude that the relationships between participation in on-campus, academic year research and subsequent pursuit of a STEM PhD are uniform among the three groups of majors. Neither the test of the joint significance of the two product terms (i.e., the χ2 of these models relative to the corresponding base model) nor the tests of the significance of the individual product terms suggest that the null hypothesis that these product terms are zero should be rejected. Moreover, the estimated increases in the probability of pursuing a STEM PhD associated with participation in on-campus, academic year research shown in the second panel of Table 4 are very nearly the same for those with other or undeclared majors as they are for those whose intended major is engineering or computer science. The estimated increase in the probability of pursuing a STEM PhD associated with participation in on-campus, academic year research is somewhat smaller for those whose intended major is one of the natural sciences as compared to the other two groups of majors, but the differences are not statistically significant.
Table 6 summarizes the results of our analysis addressing the issue of whether the relationships between participation in different types of on-campus, academic year research and subsequent pursuit of a STEM PhD are uniform. The first and third columns of Table 6 contain estimates of the coefficients of the indicator variables that represent participation in four different types of on-campus, academic year research. For each estimation sample, the null hypothesis that the coefficients of participation in different types of on-campus, academic year research are equal to each other can be rejected at the 5% level. The second and fourth columns of Table 6 contain estimates of the coefficients of the indicator variables that represent participation in the two combined types of on-campus, academic year research. The likelihood ratio for Model A relative to model B is 1.57 (Δdf=2; p=.456) for the estimation sample consisting of all non-Hispanic Meyerhoff participants and 3.54 (Δdf=2; p=.170) for the estimation sample consisting of all African-American Meyerhoff participants. Based on these test statistics, we fail to reject the joint null hypothesis that the coefficient of participation in UMBC’s annual undergraduate research symposium is equal to the coefficient of completing one research course for credit and that the coefficient of completing two or more research courses for credit is equal to the coefficient of MARC U-STAR participation. In addition, we reject the null hypothesis that the coefficients of participation in the two types of on-campus, academic year research contained in model B are equal to each other.
TABLE 6.
SELECTED RESULTS FROM ESTIMATION OF PROBIT MODELS OF DETERMINANTS OF PURSUIT OF A STEM PH.D. THAT CONSIDER DIFFERENT GROUPINGS OF ON-CAMPUS, ACADEMIC YEAR RESEARCH (STANDARD ERRORS IN PARENTHESES)
| TYPE OF ON-CAMPUS, ACADEMIC YEAR RESEARCH | ESTIMATION SAMPLE
|
|||
|---|---|---|---|---|
| All Non-Hispanic Meyerhoff Participants | All African-American Meyerhoff Participants | |||
| Model A | Model B | Model A | Model B | |
| Annual Undergraduate Research Symposium Only | −.132 (.375) | −.302 (.485) | ||
| 1 Research Courses for Credit, no MARC U-STAR | −.0258 (.314) | .261 (.353) | ||
| URCAD and/or 1 Research Semester; no MARC U-STAR | −.0715 (.250) | .0591 (.289) | ||
| 2 or more Research Semesters, no MARC U-STAR | 1.12*** (.361) | 1.54*** (.497) | ||
| MARC U-STAR | .661*** (.187) | .728*** (.198) | ||
| 2 or more Research Semesters and/or MARC U-STAR | .741*** (.176) | .817*** (.189) | ||
|
| ||||
| n | 441 | 441 | 371 | 371 |
| Log-likelihood | −226.51 | −227.29 | −186.78 | −188.55 |
| χ2 | 91.89 | 90.32 | 80.88 | 77.34 |
| χ2 Test of Null Hypothesis That the Coefficients of All Types of On-Campus, Academic Year Research Participation Included in Model Are Equal | 10.20** | 8.76*** | 8.60** | 5.67** |
p<.10;
p<.05;
p<.01
Discussion
Using data from Meyerhoff participants at UMBC, this study has addressed the relationship between participation in on-campus, academic year research and subsequent pursuit of a STEM PhD. In addition to looking at the overall relationship between participation in on-campus, academic year research and subsequent pursuit of a STEM PhD, we examined whether this relationship varies by type of intended major and whether this relationship is uniform for participation in different types of on-campus, academic year research.. The results suggest that the overall increase in the probability of pursuing a STEM PhD program associated with participation in on-campus, academic year research is substantial. Depending on how on-campus, academic year research is defined and whether the instrumented or non-instrumented estimates are used, for an average member of the non-Hispanic Meyerhoff sample, participation in on-campus, academic year research is associated with an increase of 17.1 to 29.4 percentage points in the probability of pursuing a STEM PhD. The corresponding estimates for an average member of the African-American Meyerhoff sample are somewhat larger—21.0 to 37.8 percentage points.
The preceding results are based on a probit model that controls for demographic characteristics, pre-college educational background, summer undergraduate research experiences, intended major and parental educational attainment. However, they do not address the possible endogeneity of participation in on-campus, academic year research. Using an indicator variable of whether the Meyerhoff participant was a member of one of the cohorts that had access to UMBC’s MARC U-STAR program and UMBC’s annual undergraduate research symposium as an excluded instrument, we also obtained IV estimates of the effect of participation in on-campus, academic year research. This excluded instrument appeared to work well. The resulting IV estimates were roughly consistent with our non-instrumented estimates. The IV estimates were somewhat larger and had substantially larger standard errors than their non-instrumented counterparts, however. Wu-Hausman tests fail to indicate that endogeneity is present, suggesting that the more conservative non-instrumented estimates are unbiased.
We also examined whether the relationship between participation in on-campus, academic year research and subsequent pursuit of a STEM PhD varied with intended major. As shown in Table 4, although the magnitudes of the estimated increase in the probability of pursuing a STEM PhD program associated with participation in on-campus, academic year research on pursuit of a STEM PhD are somewhat smaller for natural science majors than they are for the other two groups of majors, the variations among the increases associated with participation in on-campus, academic year research for the three types of majors are not statistically significant.
Our analysis suggests that the structure and intensity of the on-campus, academic year research experience matter. First, when we limited our definition of participation in on-campus, academic year research to completion of at least two research courses for credit or MARC U-STAR participation, the estimated coefficient of participation in on-campus, academic year research was consistently larger than it was when we also considered participation in UMBC’s annual undergraduate research symposium or completion of one research course to be a form of on-campus, academic year research. We believe these differences result from the two former types of on-campus, academic year research being generally less intense and more loosely structured than the completion of two or more research courses for credit or MARC U-STAR participation.
Additional evidence for the importance of the structure and intensity of a student’s on-campus, academic year research experience comes from the two probit models that include a series of indicator variables for participation in different types of on-campus, academic year research, rather than a single indicator variable for participation in on-campus, academic year research. Most notably, as Table 6 shows, in the model that includes four indicator variables representing participation in different types of on-campus, academic year research, the null hypothesis that the coefficients of each of these indicator variables are equal is rejected. Our results further suggest that it is reasonable to combine participation in UMBC’s annual undergraduate research symposium and completing one research course for credit and to combine completing two or more research courses for credit with MARC U-STAR participation. Finally, our results suggest that the coefficient of participating in UMBC’S annual undergraduate research symposium or completing one research course for credit is not equal to the coefficient of completing two or more research courses for credit or MARC U-STAR participation.
Although we believe our analysis makes an important contribution to understanding the relationship between participation in on-campus, academic year research and subsequent pursuit of a STEM PhD, several limitations should be noted. Most notably, with the possible exception of our instrumental-variable estimates, which have very large standard errors, the non-experimental process that generated our data limit the extent to which it can be claimed that the relationships we have estimated are causal. While randomized control trials to estimate the impacts of participation in on-campus, academic year research are, in principle, feasible, they are extremely rare (but see Nagda et al., 1998 and Hathaway et al., 2002). The importance of considering randomized control trials when designing future efforts to estimate the effects of participation in on-campus, academic year research cannot be overemphasized.
In addition, our data considered participation in three types of on-campus, academic year research. It did not, however, capture participation in all forms of on-campus, academic year research. For example, students in our sample might have volunteered in a faculty member’s lab. Since omission of participation in other forms of on-campus, academic year research would tend to bias the estimates downward, this further suggests that the estimates we have reported are conservative.
Also, the generalizability of our findings is limited. Our analyses are based on comparing Meyerhoff participants who participated in on-campus, academic year research with those who did not. Meyerhoff participants are a highly selective group with characteristics that, on average, are very different than underrepresented minority students, in general. For example, the mean high-school GPA and math SAT scores for our sample are 3.73 and 654, respectively. These are considerably higher than the national averages of 3.33 and 429 for underrepresented minorities (College Board, 2008).5 The increase in pursuit of a STEM PhD associated with participation in on-campus, academic year research might be greater or less for minority collegians with contrasting academic abilities.
Overall, we have provided evidence that participation in on-campus, academic year research is associated with an increase in the probability that minority scientists and engineers will pursue PhDs. This evidence is consistent with the work of Hathaway and colleagues (2002), even though they employed a more inclusive measure of graduate education.6 As programs for facilitating participation in on-campus, academic year research are continued and expanded, it is important to note that more structured and intense experiences appear to be of particular value, possibly as a result of skills and relationships developed that are more similar to actual graduate study.
Our results support recent national and international efforts to better understand undergraduate and graduate research that leads to STEM PhD completion. Various national researchers are currently involved in studies to determine the personal and professional gains that account for the positive relationship of undergraduate research with student success in science. (Lopatto, 2007; Frantz et al., 2006). Internationally, numerous studies attempt to improve research effectiveness by identifying and measuring research skills that students need to practice and master (Person and Brew, 2002; Craswell, 2007; Tang and Gan, 2005). However, both the national and international efforts only recently began to focus on on-campus, academic year research experiences. Therefore, the evidence from our findings provides empirical support for continuing to study whether specific gains result from on-campus, academic year research. Researchers may also be interested in understanding skills gained from on-campus, academic year research as compared to gains from other forms of undergraduate research, such as summer research internships. Thus, our research may influence global research studies on undergraduate research as well as policies and programs that facilitate these research experiences.
An important direction for future research is to examine the mechanisms by which participation in on-campus, academic year research influences STEM PhD outcomes. That is, there are likely to be a variety of cognitive and non-cognitive benefits of participation in on-campus, academic year research that explain the overall effect of such participation on students’ pursuit of a STEM PhD. Participation in on-campus, academic year research may, for example, increase excitement for the STEM fields. Another mechanism is enhancing academic records to better facilitate graduate school entry. Yet a third mechanism is enhancing proficiencies that increase success once a student is enrolled in a STEM PhD program.
Relatedly, the increase in the probability of pursuing a STEM PhD that is associated with participation in on-campus, academic year research might be due to several components, including:
an increase in the probability of college completion;
an increase in the probability of majoring in a STEM field;
an increase in the probability of applying to a STEM PhD program conditional on receiving a bachelor’s degree with a STEM major;
an increase in the probability of being accepted to a STEM PhD program conditional on applying to such a program;
an increase in the probability of enrolling in a STEM PhD program conditional on being admitted to such a program; and
an increase in the probability of, or decrease in the time to, completion of a STEM PhD conditional on enrolling in such a program.
Because different mechanisms by which participation in on-campus, academic year research influences STEM PhD outcomes might have differential effects on each of these components, examining the effect of participation in on-campus academic year research on each of these components individually might increase our understanding of the various cognitive and non-cognitive benefits of participation in on-campus, academic year research and how they contribute to the pursuit of a STEM PhD. This increased understanding of the specific effects of this type of research exposure will supply information necessary for continued support and improvement of STEM educational interventions targeted at enhancing the success of underrepresented minorities in research careers.
TABLE 5.
VALUES OF INDEPENDENT VARIABLES USED TO COMPUTE ESTIMATED INCREASES IN PROBABILITY OF PURSUING STEM PH.D. FOR DIFFERENT GROUPS OF INTENDED MAJORS
| INTENDED MAJOR
|
||||||
|---|---|---|---|---|---|---|
| OTHER/UNDECLARED | ENGINEERING/COMPUTER SCIENCE | NATURAL SCIENCES | ||||
| PARTICIPATED IN ON-CAMPUS, ACADEMIC YEAR RESEARCH?
| ||||||
| YES | NO | YES | NO | YES | NO | |
| Participation in On-Campus, Academic Year Research | 1 | 0 | 1 | 0 | 1 | 0 |
| Engineering/Computer Science Major | 0 | 0 | 1 | 1 | 0 | 0 |
| Natural Sciences Major | 0 | 0 | 0 | 0 | 1 | 1 |
| Participation in On-Campus, Academic Year Research X Engineering/Computer Science Major | 0 | 0 | 1 | 0 | 0 | 0 |
| Participation in On-Campus, Academic Year Research X Natural Sciences Major | 0 | 0 | 0 | 0 | 1 | 0 |
| All Other Ind. Variables | Sample Mean | Sample Mean | Sample Mean | Sample Mean | Sample Mean | Sample Mean |
Footnotes
Equivalently, the increase in the probit function of the probability of pursuing a STEM PhD associated with participation in on-campus, academic year research for Meyerhoff participants whose intended major was engineering and computer science is given by sum of the coefficient of participation in on-campus, academic year research and the coefficient of the product of participation in on-campus, academic year research and whether the intended major was engineering and computer science. Similarly, the increase in the probit function of the probability of pursuing a STEM PhD associated with participation in on-campus, academic year research for Meyerhoff participants whose intended major was a natural science is given by sum of the coefficient of participation in on-campus, academic year research and the coefficient of the product of participation in on-campus, academic year research and whether the intended major was a natural science.
The differences reported in this paragraph as well as the next paragraph were obtained by taking the differences between the predicted probability of pursuing a STEM PhD when:
- the value of participation in on-campus, academic year research is set equal to 1 and the values of all other independent variables are set equal to their sample means; and
- the value of participation in on-campus, academic year research is set equal to 0 and the values of all other independent variables are set equal to their sample means.
Since participation in on-campus, academic year research is not included in the reduced form equation, the issue of whether UMBC’S annual undergraduate research symposium should be considered a form of on-campus, academic year research is not germane here.
These estimates were obtained by taking the differences between the predicted probability of pursuing a STEM PhD when:
- the values of the independent variables are set equal to the values shown in the “Yes” column in Table 5 corresponding to the intended major of interest; and
- the values of the independent variables are set equal to the values shown in the “No” column in Table 5 corresponding to the intended major of interest.
The calculation of the national average math SAT score only includes the average African-Americans, the underrepresented group represented in our sample.
They considered any graduate coursework as their dependent variable.
References
- Angrist J, Imbens G, Rubin D. Identification of causal effects using instrumental variables. Journal of the American Statistical Association. 1996;91(434):444–455. [Google Scholar]
- Barlow AEL, Villarejo M. Making a difference for minorities: Evaluation of an educational enrichment program. Journal of Research in Science Teaching. 2004;41(9):861–881. [Google Scholar]
- Bauer KW, Bennett JS. Alumni perceptions used to assess undergraduate research experience. The Journal of Higher Education. 2003;74:210–230. [Google Scholar]
- Boyer Commission on Educating Undergraduates in the Research University. Reinventing Undergraduate Education: A Blueprint for America’s Research Universities. State University of New York at Stony Brook for the Carnegie Foundation for the Advancement of Teaching; Stony Brook, NY: 1998. [Google Scholar]
- Chubin D, May GS, Babco EL. Diversifying the engineering workforce. Journal of Engineering Education. 2005;94(1):73–86. [Google Scholar]
- Craswell G. Deconstructing the skills training debate in doctoral education. Higher Education Research and Development. 2007;26(4):377–391. [Google Scholar]
- Domestic Policy Council, Office of Science and Technology Policy. American Competitiveness Initiative. 2006 Retrieved on April 15, 2007 from http://www.whitehouse.gov/stateoftheunion/2006/aci/aci06-booklet.pdf.
- Frantz KJ, DeHann RL, Demetrikopoulos MK, Carruth LL. Routes to research for novice undergraduate neuroscientists. The American Society for Cell Biology. 2006;5:175–187. doi: 10.1187/cbe.05-09-0119. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Gennetian LA, Morris PA, Bos JM, Bloom HS. Constructing instrumental variables from experimental data to explore how treatments produce effects. In: Bloom HS, editor. Learning More from Social Experiments: Evolving Analytic Approaches. New York: Russell Sage Foundation; 2005. pp. 75–114. [Google Scholar]
- Gordon EW, Bridglall BL. Creating excellence and increasing ethnic minority leadership in science, engineering, mathematics and technology: A study of the Meyerhoff Scholars Program at the University of Maryland-Baltimore County. 2004 Retrieved on April 18, 2008 from http://www.ncrel.org/gap/studies/meyerhoff.pdf.
- Hathaway RS, Nagda BA, Gregerman SR. The relationship of undergraduate research participation to graduate and professional education pursuit: An empirical study. Journal of College Student Development. 2002;43(5):614–631. [Google Scholar]
- Hoffer TB, Welsh V, Webber K, Williams K, Lisek B, Hess M, Loew D, Guzman-Barron I. Doctorate Recipients from United States Universities: Summary Report 2005. Chicago: National Opinion Research Center; 2006. [Google Scholar]
- Hunter AB, Laursen SA, Seymour E. Becoming a scientist: The role of undergraduate research in students’ cognitive, personal and professional development. Science Education. 2007;91:36–74. [Google Scholar]
- Jackson SA. The quiet crisis: Falling short in producing American scientific and technical talent. 2004 Retrieved on May, 30, 2007 from http://www.bestworkforce.org/PDFdocs/Quiet_Crisis.pdf.
- Lopatto D. The essential features of undergraduate research. Council on Undergraduate Research Quarterly. 2003;XXIII(3):139–142. [Google Scholar]
- Lopatto D. Undergraduate research experiences support science career decisions and active learning. CBE-Life Sciences Education. 2007;6:297–305. doi: 10.1187/cbe.07-06-0039. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Ludwig J, Bassi LJ. The puzzling case of school resources and student achievement. Educational Evaluation and Policy Analysis. 1999;21(4):385–403. [Google Scholar]
- Maton KI, Hrabowski FA. Increasing the number of African American PhDs in the sciences and engineering: A strength based approach. The American Psychologist. 2004;59(6):547–556. doi: 10.1037/0003-066X.59.6.547. [DOI] [PubMed] [Google Scholar]
- Merkel CA. Undergraduate research at six research universities: A pilot study for the Association of American Universities. 2001 Retrieved on April 15, 2008 from http://www.aau.edu/education/Merkel.pdf.
- Miller RT, Murnane RJ, Willett JB. Do teacher absences impact student achievement? Longitudinal evidence from one urban school district. Educational Evaluation and Policy Analysis. 2008;30(2):181–200. [Google Scholar]
- Nagda BA, Gregerman JJ, Hippel WV, Lerner JS. Undergraduate student faculty research partnerships affect student retention. The Review of Higher Education. 1998;22(1):55–72. [Google Scholar]
- National Institute of General Medical Sciences. MARC program description. 2008 Retrieved June 1, 2008, from http://www.nigms.nih.gov/Training/MARC/USTARAwards.htm.
- National Science Foundation. Women, minorities and persons with disabilities in science and engineering. 2002 Retrieved April 24, 2008, from http://www.nsf.gov/statistics/nsf03312/pdfstart.htm.
- National Science and Technology Council. Ensuring a strong US scientific, technical and engineering workforce in the 21st century. 2000 Retrieved April 24, 2008, from http://www.ostp.gov/pdf/workforcerpt.pdf.
- Pearson M, Brew A. Research training and supervision development. Studies in Higher Education. 2002;27(2):135–150. [Google Scholar]
- Russell SH, Hancock MP, McCullogh J. The pipeline: Benefits of undergraduate research experiences. Science. 2007;316:548–49. doi: 10.1126/science.1140384. [DOI] [PubMed] [Google Scholar]
- Summers MF, Hrabowski FA. Preparing minority scientists and engineers. Science. 2006;311:1870–1871. doi: 10.1126/science.1125257. [DOI] [PubMed] [Google Scholar]
- Tang BL, Gan YH. Preparing the senior or graduating student for graduate research. Biochemistry and Molecular Biology Education. 2005;33(4):277–280. [Google Scholar]
- The College Board. 2007 College-Bound Seniors: Total group profile report. 2007 Retrieved June 10, 2007, from The College Board web site: http://www.collegeboard.com/prod_downloads/about/news_info/cbsenior/yr2007/national-report.pdf.
- UMBC Office of Institutional Advancement. UMBC Facts. 2008 Retrieved June 2, 2008, from the UMBC web site: http://www.umbc.edu/aboutumbc/facts.php.
- UMBC Office of Institutional Advancement. College Profile Summary. 2007 Retrieved June 2, 2008, from UMBC’s, Office of Institutional Advancement web site: http://www.umbc.edu/oir/College%20Profiles.xls.
- UMBC Office of Undergraduate Education. Undergraduate Research at UMBC. 2008 Retrieved on June 3, 2008, from UMBC’s website: http://www.umbc.edu/undergrad_ed/research/index.html.
- U.S. Census Bureau. Public Information Office. Minority population tops 100 Million. 2007 Retrieved May 1, 2008, from http://www.census.gov/Press-Release/www/releases/archives/population/010048.html.
- Wooldridge JM. Introductory econometrics: A modern approach. 4. Cincinnati, OH: South-Western College Publishing; 2009. [Google Scholar]