The Benefits of Multi-Year Research Experiences: Differences in Novice and Experienced Students’ Reported Gains from Undergraduate Research

Abstract

This mixed-methods study explores differences in novice and experienced undergraduate students’ perceptions of their cognitive, personal, and professional gains from engaging in scientific research. The study was conducted in four different undergraduate research (UR) programs at two research-extensive universities; three of these programs had a focus on the biosciences. Seventy-three entry-level and experienced student researchers participated in in-depth, semi-structured interviews and completed the quantitative Undergraduate Research Student Self-Assessment (URSSA) instrument. Interviews and surveys assessed students’ developmental outcomes from engaging in UR. Experienced students reported distinct personal, professional, and cognitive outcomes relative to their novice peers, including a more sophisticated understanding of the process of scientific research. Students also described the trajectories by which they developed not only the intellectual skills necessary to advance in science, but also the behaviors and temperament necessary to be a scientist. The findings suggest that students benefit from multi-year UR experiences. Implications for UR program design, advising practices, and funding structures are discussed.

INTRODUCTION

Widespread investment and engagement in undergraduate research (UR) demonstrate the vital role it is thought to play in educating future scientists. Federal and private funding agencies support an array of opportunities for undergraduates to participate in scientific research (National Science Foundation [NSF], 1996; National Research Council, 1999), and the number of undergraduates participating in UR in scientific fields appears to be increasing (Laursen et al., 2010); however, little is known about the trajectories of students’ development in UR experiences or the differences in outcomes between novice and experienced undergraduate researchers.

More than a decade of empirical research has demonstrated the benefits to students from participating in UR. Engaging in authentic, real-world research in science, technology, engineering, and mathematics (STEM) fields not only socializes undergraduates into scientific thinking and practices, it may also play a significant role in students’ educational and career trajectories. Many studies have demonstrated increased interest in and awareness of science careers (Ward et al., 2002; Zydney et al., 2002; Bauer and Bennett, 2003; Seymour et al., 2004; Russell, 2005; Hunter et al., 2007; Laursen et al., 2010; Thiry et al., 2011), particularly among groups underrepresented in scientific fields (Nagda et al., 1998). UR has also been argued to increase graduation rates (Nagda et al., 1998; Kim et al., 2003) and retention in their majors for underrepresented minority students (Nagda et al., 1998; Jones et al., 2010). However, the literature on the impact of UR on students’ actual career trajectories is mixed: research on UR at liberal arts colleges (LACS) has shown that participating in research serves to confirm students’ pre-existing career and educational goals (Seymour et al., 2004; Hunter et al., 2007); other studies at non-LACS have reported that UR experiences introduce the idea of graduate school and increase the likelihood that students will pursue graduate school and science careers (Kremer and Bringle, 1990; Bauer and Bennett, 2003; Lopatto 2004; Russell, 2005), particularly for students underrepresented in the sciences (Alexander et al., 1998; Hathaway et al., 2002; Villarejo et al., 2008). These divergent findings are likely due to differences in the student populations in the studies. Student populations at LACS tend to be less diverse than student populations at other types of colleges and universities; our research has shown that many liberal arts students engage in UR due to prior interest in graduate school (Seymour et al., 2004; Hunter et al., 2007). For other student populations—such as underrepresented minority students—the introduction of graduate school as a future path is an important outcome (Villarejo et al., 2008). Clearly, though, the literature indicates that UR can have a substantial impact on students’ career paths and the production of STEM majors.

Besides the possibility that UR may strengthen the scientific “pipeline,” engaging in research confers a host of intellectual, personal, and professional benefits to students. These gains both prepare students to become STEM professionals and transfer to a range of other careers and fields (Laursen et al., 2006, 2010; Thiry et al., 2011). Among other outcomes, research experiences increase students’ communication skills (Kardash, 2000; Ward et al., 2002; Bauer and Bennett, 2003; Seymour et al., 2004; Hunter et al., 2007), and critical thinking skills (Merkel, 2001; Ishiyama, 2002; Bauer and Bennett, 2003; Seymour et al., 2004; Hunter et al., 2007). UR exemplifies a “high-impact” educational practice that provides opportunities for meaningful student–faculty interactions, and access to hands-on, collaborative, analytical work (Kuh, 2008). In addition, students are more likely to gain these intellectual and scientific benefits from participating in out-of-class STEM research experiences than from other activities, such as participating in well-designed science internships or coursework with research components (Thiry et al., 2011).

The research literature offers evidence that out-of-class research experiences contribute substantially to undergraduate students’ learning and development in scientific disciplines. But do the length or intensity of students’ research experiences matter? At what point in their undergraduate careers should students enter research to maximize positive outcomes for both the student and the larger research group? A recent study of UR experiences in bioscience has suggested the beneficial effects of early entry into research on students’ academic achievement and retention in the major (Jones et al., 2010), yet we do not yet know why or how early entry (or multi-year) experiences differ from later entry (or shorter duration) UR experiences. Researchers have also not yet explored the influence of multi-year research experiences on students’ development as scientists and professionals. This paper investigates differences between first-year researchers and students with multi-year experiences and makes inferences about how these differences reflect students’ growth and development as they advance from novice to more experienced student researchers.

RESEARCH DESIGN

This mixed-methods study explored students’ scientific and professional identity development during their UR experience. We have previously reported on how, from students’ perspectives, research mentors supported students’ scientific learning and growth (Thiry and Laursen, 2011). Here we examine students’ development in UR from the vantage points of both novice and experienced student researchers. Our research questions were: What are the developmental processes through which UR students begin to display scientific traits and aptitudes? How does students’ intellectual and professional growth from UR differ for novice and experienced researchers?

The study was undertaken at two research-extensive universities in the United States, one in the south and one in the mountain west. Study participants were sampled from four different UR programs: three were housed in the western university and one at the southern university. Program 1, located at the western university, served novice research students in the biosciences. The program is a summer program with the option to continue in the following academic year. In addition to their research lab placement, students attend laboratory techniques, safety-training, and scientific communication workshops. After a full year in the program, students have the option to write a proposal to continue their research in program 2. Program 2, also located in the western university, serves experienced student researchers in the biosciences, providing summer and academic-year research experiences. Again, the research placement is enriched with some additional activities: students attend scientific communication workshops and participate in a poster session at the end of the academic year. Program 3, located at the western university, serves underrepresented minority students and first-generation college students in the biosciences. Students typically start as sophomores or juniors as novice researchers and stay in the program until the end of their senior years. Program 3 provides enrichment in the form of laboratory and safety-training workshops for novice students, as well as a journal club. Additionally, experienced students attend scientific communication workshops and receive individual educational and career counseling. All students in program 3 present their work in a poster session at the end of the academic term. Program 4, located at the southern university, serves a broad group of students in all STEM fields, with the goal of increasing diversity in STEM. Students enter program 4 as first-year students, participating in a summer bridge program before their first semester. Throughout college, students attend regular skill-building workshops, peer-mentoring and tutoring sessions, and engage in summer and academic-year research. Students also present their work at an annual culminating poster session. All four programs provide stipends. The program audiences, disciplines, and external support for students are detailed in Table 1.

Table 1.

Program elements of study sites

	Audience		Discipline		Mission	Program support
	Entry students	Advanced students	Bioscience	STEM	Increase diversity	Summer bridge	Peer mentor	Workshops and training	Journal club	Tutor	Poster session
Western university
Program 1	✓		✓					✓
Program 2		✓	✓					✓			✓
Program 3	✓	✓	✓		✓			✓	✓		✓
Southern university
Program 4	✓	✓		✓	✓	✓	✓	✓		✓	✓

RESEARCH METHODOLOGY

Our mixed methods of data collection included in-depth, semi-structured interviews and survey data from the Undergraduate Research Student Self-Assessment (URSSA; Hunter et al., 2009). Interview data helped us to understand complex behaviors, interactions, and social processes within the UR experience (Fontana and Frey, 2000), while the URSSA survey triangulated our findings using another data-collection method (Denzin, 1978). Triangulation is a social science research technique to add validity to a study by collecting data from multiple sources and using multiple methods. While quantitative research is helpful in understanding outcomes across large groups or determining what is happening, qualitative research is helpful in understanding why or how something happened. Qualitative research can also enhance our understanding of educational processes, because researchers focus on gathering data in context and exploring participants’ lived experiences (Anderson-Levitt, 2006). In other words, qualitative research provides depth and nuance to our understanding of social and cultural phenomena, such as mentoring or student engagement in research. Quantitative and qualitative research can complement each other, because they allow constructs to be explored from different angles, leading to more nuanced and complex understandings of research questions.

Many qualitative research theorists have transformed the language of traditionally quantitative concepts of research validity and reliability to make them applicable to qualitative research (Goetz and LeCompte, 1984; Janesick, 2000). These theorists have substituted terms such as credibility, appropriateness, and significance to evaluate the validity of qualitative research. We enhanced the validity of our qualitative data by aligning our research questions and constructs of interest to our interview questions, as well as situating our study within the context of prior research (Goetz and LeCompte, 1984). Reliability focuses on the replicability of a study, and replication in any educational research study can be difficult, if not impossible, because behavior and environments are always changing. However, we enhanced the reliability of this study by triangulating both our data-collection methods (using surveys and interviews) and our analytical techniques (multiple researchers coded the data and engaged in discussions about data analysis and interpretation to enhance interrater reliability).

In this study, we sought to uncover how students perceived their development of scientific traits, habits, and identities through their everyday interactions and work within research groups. We also sought to identify possible markers for stages in students’ developmental trajectories as they progressed in the research experience by exploring the differences between novice and experienced undergraduate researchers. These constructs of interest cannot adequately be addressed through external measures but must be explored through students’ own perceptions of their growth and development from the research experience. Therefore, we did not collect external data, such as advisors’ ratings of their students or graduation rates. Prior research has already addressed the influence of UR experiences on student persistence and graduation (Nagda et al., 1998; Kim et al., 2003; Jones et al., 2010). Additionally, our prior work has illustrated the ways that advisors mentor students to develop their scientific abilities (Laursen et al., 2010; Thiry and Laursen, 2011), as well as advisors’ perceptions of students’ learning and development from research (Hunter et al., 2007). Therefore, in this study, we sought students’ own impressions about their scientific and personal growth from the research experience, as well as their perceptions of the educational processes that fostered these gains.

Mixed Methodology

Qualitative Methods.

The interview protocol was designed to be exploratory in nature and to elicit detailed information about students’ perceived gains from research. The interview protocol also addressed the actual activities and work in which students engaged during their UR experiences, and their interactions with their lab advisors and research group. Interviews were conducted both in person and by telephone, lasting approximately 40 to 80 min. All interviews were taped and transcribed verbatim and then submitted to NVivo 8 software for qualitative analysis.

To analyze the interview data, we searched each transcript for information bearing upon the research questions. In this type of analysis, text segments referencing issues of different types were tagged by code names. Groups of codes that cluster around particular themes were grouped within “domains,” such as Becoming a Scientist/Professional or Personal/Professional Gains (Spradley, 1980). To determine students’ perceived gains from the research experience, we used the analytical framework of six benefits domains developed from a comparative, longitudinal study of a large sample of UR participants (Hunter et al., 2007). The domains discussed in this paper are Thinking and Working Like a Scientist, Becoming a Scientist/Professional, and Personal/Professional Gains—those most closely associated with students’ developmental trajectories. Although new codes were added to the benefits categories as necessary, no new domains were identified. Componential analysis of the qualitative codes allowed for comparisons across domain categories and demographic variables, including gender, race, or ethnicity; amount of research experience; program; or institution.

Quantitative Methods.

To complement and triangulate the interview data, we also administered the Undergraduate URSSA (Hunter et al., 2009). The URSSA is a 136-item instrument designed to enable departments and programs to gather information about what students do or do not gain from participating in UR in the sciences and about what activities contribute to those gains (Hunter et al., 2009). URSSA is grounded in extensive qualitative research identifying student outcomes from UR experiences (Hunter et al., 2007; Seymour et al., 2004). The gains scales items are rated on a 4-point Likert scale (1 = no gain, 2 = a little gain, 3 = a good gain, and 4 = a great gain). The Likert scale was developed based on pilot testing with students to assess their perceptions of constructs such as little gain, good gain, and great gain. Survey scales include the gains categories identified in our previous research (Hunter et al., 2007): Thinking and Working Like a Scientist; Personal/Professional Gains; Becoming a Scientist/Professional; Skills; Career Clarification; and Career Preparation. These scales collectively contain 47 survey items. Along with these six scales, the URSSA also collects extensive demographic data, including the extent of students’ prior research experience. Additionally, the instrument measures other aspects of the UR experience, such as students’ motivation to engage in research, the authenticity of students’ research work, the quality and quantity of advising received by students, the impact of UR on students’ educational and career aspirations, and students’ satisfaction with program support elements, such as program staff and supplemental training(s).

To develop the instrument, the URSSA was piloted and validated on multiple campuses with diverse samples of students through “think-aloud” interviews and preliminary statistical testing of items’ validity and reliability. The survey was then administered to a diverse group of more than 500 students on 22 public and private campuses for further validation and item refinement. Statistical verification of item functioning involved flagging for possible removal, alteration, or customization, and identifying items that had a preponderance of responses grouped in the upper or lower response categories, that were highly correlated with each other, or that had a majority of “N/A” responses. Some items were removed from the survey or merged with other items after this analysis. To further refine the URSSA instrument, we assessed students’ responses with confirmatory factor analysis to learn if they fit the anticipated factor structure of the survey, or if alternative structures were a better fit (Hunter et al., 2009). The URSSA met accepted standards for model fit, indicating that items functioned as anticipated after two categories from the original survey were merged.

The URSSA has met accepted measures of internal reliability. Table 2 displays sample items from the three survey scales used in this paper and reliability measures (Cronbach's alpha) for these scales. All six scales demonstrated high internal reliability (α > 0.8).

Table 2.

URSSA scale items and internal reliability analysis, measured by Cronbach's alpha coefficient

URSSA survey subscale	Number of items	Sample items (Prompt: How much did you gain in the following areas as a result of your most recent research experience?)	Cronbach's alpha coefficient (α)
Thinking and Working Like a Scientist: Applying Knowledge to Research Work	9	Analyzing data for patterns Figuring out the next steps in a research project Problem solving in general	0.92
Personal/Professional Gains	6	Confidence in my ability to contribute to science Confidence in my ability to do research	0.90
Becoming a Scientist/Professional	4	Ability to work independently Developing patience with the slow pace of research	0.84

URSSA survey data for the present study were entered into the statistical software package SPSS, in which descriptive and inferential statistics were computed. To test for differences among various subgroups of the sample, independent sample t tests, chi-square tests for goodness of fit, and two-way analysis of variance (ANOVA) were used. No statistically significant differences were found on survey scales for gender, race/ethnicity, institution, or UR program. Statistically significant differences between experienced and novice researchers were detected.

Demographic Characteristics of Study Samples

All students in the four programs (110 students) were invited to participate in a single interview during the academic years 2007–2008 or 2008–2009. All of these students were also invited to complete the URSSA. The URSSA was administered electronically, and students were sent an email invitation with a link to the survey. Those who had not completed the survey were sent weekly reminders for a month. Seventy-three students completed both an interview and the URSSA survey, for a response rate of 67%. The data discussed in this paper come from the matched set of respondents who completed both a survey and an interview.

Because two of the programs emphasized broadening participation in STEM fields, the sample represented greater gender and ethnic diversity than is typically found in STEM disciplines in the United States. Women comprised 48% of the sample, and students from underrepresented minority groups comprised 36% of the sample: 23% were African American, 12% were Hispanic, and 1% were multi-racial. The remaining students were Caucasian (47%) and Asian/Pacific Islander or Asian American (17%).

The student demographics varied by program. For instance, program 1—the program for research novices in the biosciences—had a much higher representation of women than did the other programs. Programs 1 and 2 had less racial/ethnic diversity, because diversifying STEM fields was not part of the mission of either program and students of color were typically counseled into program 3, a program supporting underrepresented minority and first-generation students. Table 3 details student demographics for each program.

Table 3.

Demographic distribution of students by UR program^a

	Number of students by UR program
Demographic variable	Program 1	Program 2	Program 3	Program 4	Total (n = 73)
Year in school
Sophomore	2	2	4	9	17
Junior	4	5	3	12	24
Senior	3	9	6	14	32
Gender
Female	8	8	7	16	39
Male	1	8	6	19	34
Race/ethnicity
Asian/Pacific Islander	2	3	1	3	9
African American	0	0	4	8	12
Caucasian	6	13	1	22	42
Hispanic	1	0	7	2	10

^aSchool year starts in September.

Students also represented a variety of disciplines, though the biosciences were heavily represented in three of the UR programs (programs 1, 2, and 3). Overall, 54% of students were biological sciences or bioengineering majors, 19% were studying biochemistry or chemistry, 7% mechanical or civil engineering, 6% computer science or computer engineering, 6% psychology, 5% physics, and 3% mathematics.

For analysis purposes, we divided the students into two groups, novice and experienced, based on their degree of prior UR experience. Prior experience includes any previous research experience, even if it was on a different project or in a different group. Students classified as novice (or entry-level) had completed two semesters or fewer of UR. For instance, a student with only one summer of UR and no academic-year experience, or a student with a summer and one semester of UR would be classified as a novice. Experienced students had completed more than two semesters plus one summer of UR. Novice students tended to be sophomores, although there was also a moderate representation of third- and fourth-year novice students (nine in each year). The sample was divided between novice researchers (40%) and experienced researchers (60%).

The Nature of Students’ Research Experiences

UR at large research universities typically takes place within research groups that may include faculty, postdoctoral scientists, and graduate students. Thus, any scientist who is more advanced than an undergraduate—including a graduate student—is considered a senior scientist. These undergraduate–postgraduate–faculty triads confer numerous benefits to all scientists involved, but they are not without challenges and tensions (Dolan and Johnson, 2010). For instance, undergraduates’ need for high-quality mentorship and professional socialization must be balanced with the time and effort required to train undergraduates and with the research group's need for productivity (Dolan and Johnson, 2010). Despite these challenges, students in this study usually had ample access to senior scientists in their research groups: URSSA findings indicate that only 10% of students—primarily novice researchers—reported infrequent contact with faculty and graduate students in their research groups. These students also reported greater dissatisfaction with their relationships with their research advisors and fewer overall gains from the UR experience. While this lack of mentoring was not a common outcome in this study, it does sound a cautionary note about the critical need for effective advising of undergraduate students, particularly novice researchers.

Despite some exceptions, most students worked under the guidance of a supportive research advisor—typically a graduate student—during their UR experiences. Research advisors supervised and guided individual students in their day-to-day research work, while faculty principal investigators (PIs) oversaw the work of the entire research team. Most students conducted research during the academic year and the summer, working anywhere from 5 to 20 h per week during the academic year, and from 20 to more than 40 h per week during the summer. Students’ time commitment often increased as they gained experience and took on responsibility. Students with 2 or 3 yr of research experience reported longer hours and more time spent in the lab than did first-year novice students.

Thus, students in this study, particularly more experienced researchers, often contributed in tangible ways to the work of the research group. Their description of their work fit what we have defined as authentic science, investigating an aspect of a genuine scientific problem of interest to the field (Laursen et al., 2010). That is, undergraduates engaged in tasks that contributed meaningfully to advancing the work of the research group and in which they had an intellectual investment. This contrasts with necessary but menial work, such as washing dishes or doing rote technical tasks without a deeper understanding of the research problem. In other words, undergraduates are treated as valuable members of the research group, and not simply as assistants to accomplish menial tasks.

In interviews, novice students described a steep learning curve as they were introduced to new research techniques and became familiar with the concepts underlying their projects. But with guidance, support, and practice, many students advanced to a point at which they analyzed data, interpreted findings, deliberated about the next steps of an experiment, and contributed input to experimental designs and methods. Occasionally, experienced students engaged with the larger scientific community by presenting their findings at professional conferences (eight students, 11% of the sample) or coauthoring journal articles (seven students, 9% of the sample).

FINDINGS

We describe our findings in terms of three gains categories identified in our previous research on UR that are most closely linked to students’ developmental trajectories: Thinking and Working Like a Scientist, Personal/Professional Gains, and Becoming a Scientist/Professional (Hunter et al., 2007). To our surprise, there were few to no qualitative or quantitative differences in students’ reports of their outcomes from the UR experience when examined along variables that we expected would be salient, such as gender or race/ethnicity. In interviews, women and students from underrepresented minority groups discussed gains in confidence and interest more often than their majority peers (Thiry and Laursen, 2011), although there were no significant differences on the URSSA scales. Lopatto (2004) also found few differences in UR outcomes based on variables such as gender or race/ethnicity. Rather, the extent of students’ prior research seems to be the most important variable in examining differences in students’ outcomes (see Tables 5, 8, and 11 later in the paper).

Table 5.

Disaggregated means and SDs on URSSA Thinking and Working Like a Scientist scale^a

Class rank	Research experience	Scale mean	SD
Sophomore	Novice	2.95	0.61
	Experienced	2.92	0.33
Junior	Novice	3.38	0.12
	Experienced	3.47	0.57
Senior	Novice	3.18	1.1
	Experienced	3.50	0.68

^aExperienced students: n = 44; novice students: n = 29. Overall scale mean = 3.35; overall SD = 0.65.

Table 8.

Disaggregated means and SDs on URSSA Personal/Professional Gains scale^a

Class rank	Research experience	Scale mean	SD
Sophomore	Novice	2.80	0.86
	Experienced	3.02	0.72
Junior	Novice	2.87	1.0
	Experienced	3.45	0.53
Senior	Novice	2.90	1.2
	Experienced	3.35	0.65

^aNovice students: n = 29; experienced students: n = 44. Overall scale mean = 3.23; overall SD = 0.72.

Table 11.

Disaggregated means and SDs on the URSSA Becoming a Scientist/Professional scale^a

Class rank	Research experience	Scale mean	SD
Sophomore	Novice	2.84	0.99
	Experienced	3.15	0.71
Junior	Novice	3.00	0.35
	Experienced	3.66	0.57
Senior	Novice	2.87	1.0
	Experienced	3.55	0.64

^aNovice students: n = 29; experienced students: n = 44. Overall scale mean = 3.42; overall SD = 0.71.

Student responses on the URSSA corroborated our findings from the qualitative data, demonstrating that the URSSA can measure indicators that reflect complex theoretical constructs, such as scientific identity development and epistemological growth. The instrument detected statistically significant differences between experienced and novice researchers, suggesting that the URSSA is sensitive to differences in student outcomes.

Thinking and Working Like a Scientist

The analytical category Thinking and Working Like a Scientist refers to students’ gains in applying scientific knowledge and skills, understanding the scientific research process, and improving their conceptual understanding of the field. Epistemological growth, or gains in understanding the nature of scientific knowledge, is also an essential aspect of students’ cognitive development from research.

Chi-square tests for goodness of fit on the URSSA scales demonstrate the cumulative benefit to students from multi-year participation in UR. Using Pearson's chi-square test, we found a significant effect for multi-year research experience on the Thinking and Working Like a Scientist scale, χ²(2, N = 70) = 8.30, p = 0.016. Experienced students rated their gains as higher on items such as analyzing data for patterns, problem solving, and identifying flaws in the interpretation of data. Table 4 details the individual item means and SDs for the Thinking and Working Like a Scientist scale of URSSA.

Table 4.

Individual item means and SDs for URSSA Thinking and Working Like a Scientist scale^a

	Novice students		Experienced students		All students
Item. How much did you gain in the following areas as a result of your most recent research experience?b	Mean	SD	Mean	SD	Mean	SD
Understanding how to collect scientific data	3.40	0.64	3.61	0.65	3.56	0.65
Understanding how scientific research is done	3.43	0.79	3.71	0.55	3.64	0.61
Analyzing data for patterns	3.10	0.76	3.35	0.79	3.30	0.79
Interpreting results from analyzing scientific data	3.09	0.86	3.40	0.74	3.33	0.77
Problem solving in general	3.15	0.77	3.44	0.76	3.37	0.77
Formulating a research question that can be answered with data	3.26	0.75	3.21	0.93	3.23	0.89
Identifying limitations in research methods and designs	3.19	0.72	3.41	0.74	3.36	0.72
Identifying flaws in the interpretation of data	3.09	0.76	3.35	0.83	3.31	0.81
Figuring out the next steps in a research project	3.17	0.79	3.24	0.88	3.22	0.85

^aExperienced students: n = 44; novice students: n = 29. Overall scale mean = 3.35; overall SD = 0.65.

^b1 = no gain; 2 = a little gain; 3 = good gain; 4 = great gain.

A two-way ANOVA was conducted to examine the effect of prior research experience and year in school on students’ outcomes on the Thinking and Working Like a Scientist scale. There were no significant main effects for prior research experience (F(1,65) = 0.396, p = 0.532) or year in school (F(1,65) = 2.507, p = 0.089). Additionally, there was no significant interaction between the effects of prior research experience and year in school (F = 0.284, df = 2, p = 0.754). The model may not have had enough statistical power to detect significance, as observed power was only 48%, indicating the sample size may have been too small to detect statistical significance.

Table 5, however, suggests differences in intellectual outcomes between novice and experienced researchers among seniors, and to a lesser degree, juniors. In fact, Tukey's post hoc tests detected significant mean differences between results for sophomores and juniors (p = 0.038) and sophomores and seniors (p = 0.026). In other words, seniors report more intellectual benefits from prior research experience than do sophomores. Given the disparity in mean scale scores between novice and experienced seniors, the data suggest that senior year may be too late for students to enter research in order to achieve the greatest benefit. We can tentatively conclude that students gain more intellectual benefits from multi-year research experiences. However, further statistical testing with larger sample sizes is needed.

Qualitative findings shed further light on the differences in novice and experienced students’ perceptions of their cognitive development and thus begin to outline the developmental progression of undergraduates’ intellectual growth from research. As shown in Table 6, experienced students in interviews were more likely to report gaining higher-level scientific skills, while novice students were more likely to discuss gains in mastering data-collection techniques. Almost all students—novice and experienced—described gains in collecting scientific data, while only 14% of novices and fully 50% of experienced students reported gains in advanced scientific skills, such as understanding experimental design.

Table 6.

Frequencies of Thinking and Working Like a Scientist codes from student interviews^a

	Novice students		Experienced students
Code	n	% of novices	n	% of experienced students
Gains in data-collection skills	27	93	44	100
Gains in data analysis and interpretation	12	41	28	64
Gains in problem solving	13	45	42	95
Gains in figuring out the next steps of an experiment	3	10	12	27
Gains in understanding experimental design	4	14	22	50
Gains in identifying a research question	1	3	7	16

^aNovice students: n = 29; experienced students: n = 44.

In interviews, students were asked what contributed to their understanding of the science underlying their research project. Many students responded that they did not have the conceptual background to understand their project when they first entered the research group. Early entry novice students (typically sophomores), in particular, had not taken the advanced courses in their major that might help them to understand the theoretical underpinnings of their projects or the significance of their work within the broader discipline. Novice students who regularly interacted with senior scientists, participated in group meetings, and conducted reviews of relevant literature began to develop a better understanding of the essential concepts in their field. The following comment is typical of how novices described learning basic techniques and concepts from their research advisors through regular lab interactions:

I've done quite a bit of molecular biology with [my advisor], actually preparing DNA, and RNA, and making gels. That's really a fun experience, ’cause he's an excellent molecular biologist. So we just hang out, and he explains to me all about how the DNA and the RNA work, and how you have to be careful because you can contaminate the room with DNA, and you can't ever get it out. And so I've been learning lots of the—I don't actually know all the names of them—but I have been learning a lot of molecular biology techniques.—Novice student

Experienced students often described a process in which their understanding of difficult concepts or terminology had “just clicked” at some point during their research experience. Subsequently, they were not only able to better understand the science underlying their experiments but also to think about how their new knowledge might transfer to future situations. Feldman et al. (2009) describe this growth as students’ cognitive progression from novice researchers to proficient technicians, although they were applying these terms to graduate students. The following comment is typical of the way that experienced undergraduate students described the progression of their conceptual understanding:

I think it takes time to develop, to figure out what exactly you're studying. You get introduced to it on such a small level that you're like, “I don't even know what that is,” and so you have to read and talk about it, and then it comes with time. It's weird because I feel like it just happened, and one day I was like, “Now I know what they're talking about!”—Experienced student

Once students had gained basic knowledge of the concepts underlying their research project, they could then begin to apply that knowledge to a research question or problem. Mastery of key concepts and theories, and the development of critical-thinking and problem-solving skills, were prerequisites for developing more advanced scientific thinking skills, such as determining the next steps of an experiment or designing an experiment. While almost all students reported gains in data-collection techniques, both in interviews and on the URSSA, experienced students were more likely than novice students to have engaged in data analysis. Fully 66% of experienced students reported growth in this area in interviews, while only 41% of novice students reported having participated directly in analyzing and interpreting their data. The following comment from an interview is representative of the ways in which students described gaining data analysis and interpretation skills:

At first [analyzing data] was hard. You could see there's an effect, but it was hard to see what that effect was, or what contributed to it. But, after reading through several times with the grant proposal, and looking at a lot of different graphs, and then our weekly meetings—now I can, at least within our lab, definitely look at a graph or table, and be able to infer that there was an effect, and what that effect was from. At first it was hard, but now it's definitely much easier.—Experienced student

Few prior studies have reported undergraduate gains in experimental design skills (Kardash, 2000; Hunter et al., 2007; Feldman et al., 2009). Likewise, in this study, only the most experienced students described engaging in these higher-order activities. Only four novice students reported gaining experimental design skills in interviews, and no novice students reported a “great” gain in this area on the URSSA. In interviews, experienced students expanded on the processes by which they began to develop experimental design skills. Students learned from participating in discussions about research design in lab meetings or by providing input into the design of their own project. One student had designed her own experiment independently, while the other students reported that they had designed or modified experiments in close collaboration with a senior scientist, as described in the following observation:

I had input [into the experimental design]. I looked at a couple of papers that did similar work and then brought my ideas up to [the PI and my research advisor]. And they agreed and disagreed with a few things, and so we changed and found what works best. It was a joint effort. I brought up the idea and they were just like, “Oh yeah, that's definitely something that we were hoping to get done.”—Experienced student

In conclusion, novice and experienced students displayed distinct differences in the nature of their reported intellectual gains that suggest patterns in cognitive developmental trajectories as students advanced in research. Novice undergraduate researchers first gained basic conceptual and disciplinary understandings of the field and mastered data-collection techniques, and then began to develop problem-solving skills as they confronted setbacks and learned to troubleshoot experiments. Experienced undergraduate researchers gained further skills in analyzing and interpreting data and honed their problem-solving skills. More rarely, some very experienced students gained advanced scientific thinking skills, such as proposing a research question or providing input into developing or modifying an experimental design. As we have reported before (Laursen et al., 2010), these advanced skills came from greater exposure and personal experience in research. The social interactions within the research experience were integral to students’ intellectual gains. Students’ regular interactions with senior scientists and their discussions with their mentors and research groups about their work and the research literature helped them to gain these new skills and understandings (Thiry and Laursen, 2011).

Personal/Professional Gains

As students became more intellectually and socially integrated into their research groups, they began to gain confidence in their ability to do research and to feel that they could make a concrete contribution to the work of the group and to their discipline overall. Differences for novice and experienced researchers on the Personal/Professional Gains scale of the URSSA were not statistically significant using Pearson's chi-square test, indicating that both novice and experienced students report substantial personal and professional gains from research. The qualitative data, however, reveal more nuanced differences in the nature of these gains.

Unlike the intellectual gains, which demonstrate a clear progression, the gains in the Personal/Professional Gains scale focus on confidence and collaboration—benefits that seemingly can be achieved much earlier in the research experience. Table 7 displays the means and SDs of the individual items on the Personal/Professional Gains scale, disaggregated by experience level.

Table 7.

Individual item means and SDs for URSSA Personal/Professional Gains scale^a

	Novice students		Experienced students		All students
Item. How much did you gain in the following areas as a result of your most recent research experience?b	Mean	SD	Mean	SD	Mean	SD
Confidence in my ability to do research	2.82	1.1	3.38	0.68	3.26	0.81
Confidence in my ability to contribute to science	2.75	1.0	3.32	0.78	3.19	0.87
Comfort in discussing scientific concepts with my research mentor	3.18	0.60	3.40	0.82	3.33	0.76
Comfort in discussing scientific concepts with other research students	2.85	0.84	3.45	0.69	3.31	0.76
Comfort in working collaboratively with others	3.09	0.70	3.64	0.57	3.47	0.65
Confidence in my ability to do well in future science courses	2.98	0.94	3.26	0.79	3.19	0.83

^aNovice students: n = 29; experienced students: n = 44. Overall scale mean = 3.23; overall SD = 0.72.

^b1 = no gain; 2 = a little gain; 3 = good gain; 4 = great gain.

Differences between novice and experienced students emerged when examining the data in light of students’ year in school. A two-way ANOVA was conducted to examine the effect of prior research experience and year in school on students’ personal and professional outcomes. There was a marginally significant main effect for prior research experience (F(1, 63) = 3.583, p = 0.063), a nonsignificant main effect for year in school (F(1, 63) = 0.525, p = 0.594), and no significant interaction between the effects of research experience and year in school (F(2, 63) = 0.229, p = 0.796). The model may not have had enough power, as observed power was 50%, indicating the sample size may have been too small to detect statistical significance.

Table 8, however, reveals differences in personal and professional outcomes based on prior research experience for students at all levels—sophomores, juniors, and seniors. The mean difference in Personal/Professional Gains scores between experienced and novice sophomores was 0.22, while it was 0.58 for juniors and 0.45 for seniors. Although novice students gain confidence and team skills from research, experienced students consistently report larger gains, regardless of their year in school. Table 8 presents the scale means and SDs for the Personal/Professional Gains scale disaggregated by year in school and prior research experience.

Even though novice students rated their personal gains lower on the URSSA, in interviews, both novice and experienced students described gaining confidence from their research experience. However, the nature of their gains in confidence differed. Table 9 outlines the coding frequencies for codes generated in the Personal/Professional Gains domain.

Table 9.

Frequencies of Personal/Professional Gains codes from student interviews^a

Code	Novice students		Experienced students
Gains in:	n	% of novices	n	% of experienced students
Confidence in ability to succeed in the discipline	3	10	1	2
General confidence	9	31	3	7
Confidence in ability to do research	8	28	5	11
Increased professional confidence	8	28	14	32
Confidence in ability to contribute to science	7	24	19	43

^aNovice students: n = 29; experienced students: n = 44.

When asked in an open-ended manner about gains from the research experience, novice students were more likely to discuss general self-confidence, while experienced students were more likely to describe gains in their professional confidence and in their ability to contribute to science. Novice students became more comfortable in the lab and gained general confidence from interacting with scientists, as reflected in the following typical comment:

I felt pretty uncomfortable when I started, because it was a really new situation for me, and it was outside of what I had done before. And so now I feel a lot more comfortable approaching people who have PhDs and master's [degrees], and asking them about their work, ’cause I've worked with them now.—Novice student

It was important to students, particularly those just entering a research group, to feel comfortable with scientists in their group and to be taken seriously by them. Again, this highlights the need for senior scientists to provide plenty of guidance and support to undergraduates in the lab, especially novice students.

While novice students were beginning to feel comfortable being around scientists, experienced students were beginning to see themselves as scientists. When students felt they were perceived as having something to offer scientifically, they gained confidence in their research abilities. Experienced students often described being treated like a colleague in their research groups.

The research advisor I have now is really good about sitting down with you and explaining, “Hey, look, this is why we're doing this.” I think in the research experience, she kind of looks to me, not as a peer per se, but at least like I have something to offer. I've probably not done as much, but they treat me more as a relevant source and not just an undergrad.—Experienced student

Experienced students also discussed the personal benefits of feeling that they had contributed to science. Students found great reward in the process of discovery and generating new knowledge. Fewer novice students had progressed enough on their project to recognize this gain, but some experienced students had generated results that could potentially advance knowledge in the field and subsequently had gained confidence in their scientific and professional abilities:

I have some preliminary data that is the first to ever show this particular thing. So that was really rewarding, to be able to go up to [the PI's] office and say, I think I might be the first person in the world to have found out something.—Experienced student

In conclusion, both novice and experienced students described gains in confidence from their research experiences as reported in survey and interview data. As students gained responsibility and independence within their research groups, the nature of their assertions in interviews changed. Students illustrated the processes of their personal and professional growth. Undergraduates advanced from simply feeling comfortable being around scientists, to feeling that they, too, could be a scientist and make a contribution to their field. Students’ perception that their work was valued within their research groups was an important aspect of this transition in identity. As with intellectual gains, students’ personal and professional gains resulted from positive and productive interactions with members of their lab groups and from taking on increasing challenge and responsibility in their research work.

Becoming a Scientist/Professional

Through participating in research, students began to adopt the behaviors, temperament, and values necessary to become a scientist. Students gained the ability to work independently and took on greater ownership for a project. They also began to develop patience and perseverance in the face of setbacks or failures on their project. Using Pearson's chi-square test, we found a significant effect for multi-year research experience on the Becoming a Scientist/Professional scale, χ²(2, N = 71) = 29.027, p = 0.034. Table 10 presents the means and SDs of the individual items on the Becoming a Scientist/Professional scale.

Table 10.

Individual item means and SDs for URSSA Becoming a Scientist/Professional scale^a

	Novice students		Experienced students		All students
Item. How much did you gain in the following areas as a result of your most recent research experience?b	Mean	SD	Mean	SD	Mean	SD
Ability to work independently	2.86	0.97	3.46	0.69	3.31	0.80
Developing patience with the slow pace of research	2.80	0.80	3.38	0.68	3.25	0.74
Understanding what everyday research work is like	3.25	0.82	3.69	0.52	3.60	0.62
Taking greater care in conducting lab procedures	3.25	0.56	3.65	0.60	3.57	0.61

^aNovice students: n = 29; experienced students: n = 44. Overall scale mean = 3.42; overall SD = 0.71.

^b1 = no gain; 2 = a little gain; 3 = good gain; 4 = great gain.

When examining students’ scientific identity gains in light of their year in school, it appears that prior research experience matters in students’ identity development. A two-way ANOVA was conducted to examine the effect of prior research experience and year in school on students’ scientific identity outcomes. There was a significant main effect for prior research experience (F(1, 64) = 8.587, p = 0.005). No significant interaction was detected between the effects of research experience and year in school (F(2, 64) = 0.540, p = 0.585). As with other URSSA scales, experienced researchers rated their gains substantially higher than novice students, regardless of year in school. Even sophomores, who generally demonstrated lower mean differences between novices and experienced students in their self-ratings on URSSA scales, reported a stronger scientific identity if they had spent more time on research. Table 11 presents the disaggregated means and SDs on the Becoming a Scientist/Professional scale.

In interviews, students described their growth in science identity and provided some insight into the ways that undergraduates begin to develop (or not) a conception of themselves as a scientist from their research experiences. Table 12 illustrates the frequencies for the codes generated in the Becoming a Scientist/Professional domain.

Table 12.

Frequencies of Becoming a Scientist/Professional codes from student interviews^a

Code	Novice students		Experienced students
Gains in:	n	% of novices	n	% of experienced
Ability to work independently	16	55	26	59
Developing patience with slow pace of scientific research	11	38	19	43
Developing ownership of a project	6	21	28	64
Developing greater responsibility	5	17	18	41

^aNovice students: n = 29; experienced students: n = 44.

Both novice and experienced students discussed developing temperamental characteristics, such as patience and perseverance, that would help them in a scientific career. Students learned to persevere when their experiments did not go as planned and to be careful and meticulous about research procedures. Novice students had to reconcile the fit between their own personality and the uncertain and slow nature of scientific research, as described by numerous students:

I'm not a very patient person, and so I think that it taught me to be patient. And to realize that it's better to do something accurately, even if you have to do it a little bit slower, than to rush through everything, and screw it up.—Novice student

Students also learned about the nature of scientific research work—that it is a slow, lengthy process with inherent setbacks and failures. Novice students, however, were not always able to generalize their own experiences to the research process in general. For example, one novice student described her particular project as “finicky,” meaning prone to errors or failure. In ascribing the fallible nature of the research process in general as something unique about her own project, she demonstrated that she was not yet able to locate her own experiences within the broader context of the scientific research process. As students gained research experience, they became more comfortable with the time that it took to do careful research.

Most students also noted that their ability to work independently grew as they gained experience, although experienced students (95%) reported gains in independence more often than novices (62%) on the URSSA. In interviews too, the way they described their developing independence differed: novice students tended to discuss learning laboratory techniques from their advisors, then gaining independence to implement them without supervision:

And it's kind of nice—when I learn something new I work directly with [my advisor]. And then at some point he allows me to kind of have the autonomy to do things on my own, and then follow up with him on it. So it's kind of nice that they're trusting me to do work on my own.—Novice student

Experienced students, in contrast, often discussed scientific independence not only in terms of conducting technical procedures, but also in thinking through problems or findings. The following comment is typical of the way in which experienced students began to develop greater intellectual independence:

I think I've definitely grown from where I was. I've definitely improved because I used to just think that I would be led through what to believe, and what conclusions to have and stuff. And now I know that I can come up with those myself, by really analyzing what went wrong in the procedure.—Experienced student

Thus, students saw gains in certain temperamental characteristics, such as patience, independence, and initiative, that are necessary to become a successful researcher. Novice students needed to assess the fit of their personality with the nature of research work; students who felt their temperament fit the scientific research process began to see themselves as “scientists.” The nature of students’ independence also differed, as novice students became more independent in conducting research procedures, while experienced students became more independent in scientific thinking and problem solving. As students described their identity development, they also noted the processes by which they began to identify as a scientist. Similar to other aspects of their growth, students mentioned that regular interactions with senior scientists, being treated as a valuable member of the team, and gaining responsibility and independence in their work all contributed to their identity development as a scientist.

Limitations of the Study

This study was exploratory in nature; our intention was to investigate the messy process of students’ scientific and personal growth from research experiences, particularly in terms of their developmental trajectories. Due to this descriptive and exploratory focus, the study was not designed to be generalizable. Further research with large national samples of UR students is needed to test the hypothesis that experienced students gain more from research than novice students, regardless of year in school, age, or other variables.

DISCUSSION AND IMPLICATIONS

Our findings identify differences in the cognitive, personal, and scientific development described by novice and experienced student researchers in interviews and reported on the URSSA. These differences illustrate undergraduates’ scientific growth and suggest patterns in developmental trajectories as they spend more time conducting research. Novice researchers reported developing basic cognitive skills, such as mastering data-collection techniques or gaining greater understanding of the process of scientific research, similar to the level of mastery of novice researchers described by Feldman et al. (2009). Novice undergraduate researchers also made general gains in confidence as they became more comfortable conducting research work and interacting with scientists within the research group. Novice students began to develop elements of a scientific temperament as they gained patience in the face of setbacks or failures in research and came to understand the slowness and ambiguity of scientific research.

Experienced undergraduate researchers built upon this foundation and began to develop higher-order scientific thinking skills, such as the ability to analyze and interpret data, solve problems, or identify the next steps in an experiment. In the terminology of Feldman et al. (2009), most of these students had become proficient technicians. In other words, they had achieved technical, but not yet intellectual, independence. A few undergraduates—typically students with 3 or 4 yr of experience on a research project—had designed an experiment with guidance from their research advisor. Approximately 10% of students published or presented research findings that contributed to the body of knowledge in their field. These few students had begun to take the first tentative steps toward becoming knowledge producers (Feldman et al., 2009). As they advanced in the research experience, experienced researchers took more initiative and demonstrated greater intellectual independence, responsibility, and ownership of their work. Experienced researchers were more likely to feel that their contributions advanced the work of their research groups. Perhaps most importantly, experienced students gained confidence in their abilities as scientific researchers and came to feel that they could be scientists. These findings suggest that multi-year research experiences may help to cement students’ commitment to their fields and increase their confidence that they can succeed in science.

While previous research has clearly identified the numerous benefits to students from participating in UR, it has not elucidated the sequence of scientific growth for undergraduate researchers. Greater understanding of students’ intellectual, personal, and professional growth from research has important implications for the mentoring and supervision of undergraduate researchers. Many research advisors who work with undergraduates have not thought about the educational or pedagogical implications of their advising practices; they simply advise students as they were once advised (Feldman et al., 2009). Yet student reports in interviews about the importance of group meetings and positive interactions with senior scientists suggest the critical role that high-quality mentoring and advising play in contributing to students’ research gains (Thiry and Laursen, 2011). Moreover, differences between novice and experienced undergraduate researchers are not only intellectual, but personal and temperamental as well. Preparation of mentors for UR could incorporate more information about the differences between novice and experienced students and the sequence of students’ growth from a research experience. Mentor training could also include recommendations for how to scaffold projects to meet students’ developmental needs and guidance in how to best support novice research students. Advisors with a better understanding of these clearly pedagogical aspects of their work with undergraduates (Laursen et al., 2010) may be better able to adjust their strategies to meet undergraduates’ needs along the developmental continuum.

Our findings also affirm the educational advantages of multi-year UR experiences. Though students who begin UR as first- or second-year college students often faced a steep learning curve to become familiar with concepts and techniques during their first year of research, their effort—and that of their research advisors—paid off during subsequent years. During their junior and senior years, experienced researchers demonstrated distinct cognitive, personal, interpersonal, and professional differences from novices, and displayed greater progress on the path of science identity development. Clearly, early entry into UR that enables multi-year research experience maximizes students’ learning and development in STEM disciplines—as long as students are adequately guided and supported by their research advisor and research group.

Yet students’ early entry into UR also carries costs to faculty and other senior scientists, who must design research projects suitable for beginners and invest significant time and resources to get students to the point at which they can be productive researchers (Dolan and Johnson 2009, 2010; Laursen et al., 2010). Undergraduate students may not “work out” or may leave the research group just as they become productive. Additionally, the authenticity of student experiences is inextricably linked to the fact that students are doing “real science” in which their research advisors are intellectually invested (Laursen et al., 2010). Thus, faculty engagement and autonomy are important drivers of UR opportunities and experiences, just as adequate supervision and mentoring within the research group are integral to undergraduates’ learning and developmental processes from research (Thiry and Laursen, 2011).

Another tension is that many funding structures in the United States do not currently support extended research experiences for undergraduates. For example, the Research Experiences for Undergraduates (REU) program funded by the NSF is designed to offer single, short-term research opportunities to many students, maximizing the number of students who may benefit from the opportunity but constraining multi-year experiences. Our findings about the educational benefits of multi-year research experiences to students’ learning and socialization into the scientific profession must therefore be balanced against an investment of time and effort by the research group to train multi-year undergraduate researchers that is unsupported by current funding structures. Given the potential benefit to students from high-quality, extended research experiences, individual UR programs, funders, and science faculty must consider how to optimize the design and structure of UR experiences in light of other concerns.