Measuring Networking as an Outcome Variable in Undergraduate Research Experiences

This paper proposes, presents, and validates a simple survey instrument to measure student conversational networking. The tool covers both personal and professional social networks and may be significant for exploring issues of retention and the development of scientific literacy.

Abstract

The aim of this paper is to propose, present, and validate a simple survey instrument to measure student conversational networking. The tool consists of five items that cover personal and professional social networks, and its basic principle is the self-reporting of degrees of conversation, with a range of specific discussion partners. The networking instrument was validated in three studies. The basic psychometric characteristics of the scales were established by conducting a factor analysis and evaluating internal consistency using Cronbach’s alpha. The second study used a known-groups comparison and involved comparing outcomes for networking scales between two different undergraduate laboratory courses (one involving a specific effort to enhance networking). The final study looked at potential relationships between specific networking items and the established psychosocial variable of project ownership through a series of binary logistic regressions. Overall, the data from the three studies indicate that the networking scales have high internal consistency (α = 0.88), consist of a unitary dimension, can significantly differentiate between research experiences with low and high networking designs, and are related to project ownership scales. The ramifications of the networking instrument for student retention, the enhancement of public scientific literacy, and the differentiation of laboratory courses are discussed.

MEASURING NETWORKING AS AN OUTCOME VARIABLE IN UNDERGRADUATE RESEARCH EXPERIENCES

Recent educational assessment research has focused on a series of interactional, psychosocial variables that have the potential to aid in exploring the particular characteristics of course-based undergraduate research experiences (CUREs; Chemers et al., 2011; Estrada et al., 2011; Auchincloss et al., 2014; Hanauer and Dolan, 2014; Corwin et al., 2015). The aim of this assessment work is to be able to provide measures that may enhance the ability of program designers, instructors, administrators, and researchers to explain how CUREs function and what outcomes emerge for students who participate in courses of this type. In particular, there is a desire to explore issues of retention, minority student inclusion in the sciences, and development of science literacy levels across society. Measures such as project ownership, self-efficacy, science identity, and scientific community values have been used to understand the relationship of these variables to aspects of laboratory course design, student experience, and student retention in the sciences.

To date, the understanding of psychosocial variables in this context has primarily related to the psychology of the individual student. For example, issues such as project ownership or scientific community values involve an interaction with a scientific environment, but the measure is of the result of this interaction in the self-perception of the student. There is, however, an additional way of looking at psychosocial contextualization. Perhaps it is important to learn something about the ways in which students interact with other people in their surroundings. What networks emerge as a result of the taking part in a CURE? Or on an even simpler level, who is the young researcher actually talking to while participating in a research experience? While this may seem very basic, potentially, this is an important issue. The scientific community is constructed primarily through language and often through direct, face-to-face communication. Simply put, talking with others is an important aspect of science (Lemke, 1990; Osbourne, 2002; Hanauer, 2006).

In a recent paper defining the features of a CURE, collaboration was situated as a basic component of this type of educational experience (Auchincloss et al., 2014). Specifically, three levels of collaboration were addressed in defining a CURE: 1) peer−peer group work, 2) interaction within a team of scientists, and 3) broad interaction with the scientific community. The importance of collaboration in a CURE and its associated requirement for actually talking to others is that it reflects how science is conducted in professional settings. In terms of scientific activity, scientists meet to discuss, share, interpret, and argue about the meaning and conclusions concerning their research and, ultimately, what this research says about the natural world (American Association for the Advancement of Science, 1993; National Research Council, 1996). Science is very much a communal activity in which scientists talk to one another and discuss, present, publish, and argue their positions. Accordingly, mutual discussion with other scientists within the laboratory, at meetings, and across the community is a crucial aspect of the development of scientific knowledge. On a similar level, science education could not take place without extensive personal interaction and talking about science. Osbourne (2002) states this in the following way: “The central goal of science education is to help students to use the language of science to construct and interpret meaning” (p. 208). From an educational perspective, science is a very specialized discourse that has to be learnt, and talking is one linguistic mechanism through which this can happen. Finally, talking about science is important within the broader societal context. Worries over a disconnect between scientists and the broader public have led to various initiatives, such as the National Science Foundation’s 2009 “Year of Science,” which involves engaging the broader public in conversations about science. Scientists talking to journalists, politicians, administrators, parents of students, and potential students has become a priority in an attempt to overcome disengagement with science in the broader public (Miller, 2004; National Science Board, 2008).

The aim of the current paper is to continue the work already conducted in the development of appropriate assessment outcome measures for CURE programs by proposing, developing, presenting, and validating a simple survey instrument to measure student conversational networking. The basic principle of this tool is the self-reporting of degrees of conversation with a range of specific discussion partners. In relation to a series of enlarging social circles, students are asked to self-report on whether they are talking to their parents, friends, students at other institutions, or professors beyond their course instructors. To an extent, the networking scales explore the degree to which laboratory course research reaches beyond the classroom. It is important to remember that a CURE is defined by its aim of reaching the scientific community and having social relevance. Accordingly, conversation should be happening with parties who are within both the existing personal and emergent scientific community frames. Evidence dealing with conversations that students are having concerning their research should allow some insight into both the emergence (or not) of a professional network of peer and more senior scientists and the potential social impact of research being conducted. As with other new instruments, it is important to have appropriate and explicit information addressing the validity and reliability of the tool. The current paper addresses this need and is organized around the following research questions:

1. What are the psychometric properties of the networking instrument?

2. Does the networking instrument discriminate between different undergraduate research experiences?

3. Is there a relationship between the networking scales and the more established psychosocial variable of project ownership?

METHODS

Overall Design

For evaluation of the psychometric properties and assessment value of the networking scales, a series of three different research designs were used. As an initial stage, an investigation of the basic psychometric characteristics of the scales was conducted. In accordance with established conventions of assessment tool development, the internal structure and dimensionality of the networking scales were analyzed using a factor analysis approach. The internal consistency of the tool was calculated using Cronbach’s alpha. Following the establishment of the core characteristics of the networking scales, a second study was conducted to explore whether the networking scales could differentiate between different undergraduate research laboratory experiences. This study involved comparing networking scales outcomes between two different undergraduate laboratory courses. The comparison consisted of a CURE and a traditional laboratory experience. The particular CURE chosen for this study involved a program situated across a range of universities and offered the opportunity for a series of networking possibilities, making it a likely candidate for promoting networking.

The final study conducted was different in its validation approach and involved looking at potential relationships between specific networking items and established psychosocial variables used to evaluate undergraduate research experiences. Underpinning this approach to validation was the idea that networking, as defined here in terms of the discussion of personal research, should be influenced by the actual educational experience of the laboratory course. In particular, the level of connectedness students felt toward their research and the degree of emotional engagement with the course should result in increased discussion of this research with other people. Simply put, if the CURE proved to be personally relevant and managed to elicit positive emotional responses, then this should increase the probability of the student discussing this research with others. Within the literature on interactional, psychological outcomes for students who have participated in a CURE, the variable that has come closest to providing data on the self-positioning of the student in terms of the educational experience, personal connection to the research project, and emotive responses has been project ownership. Research has been conducted to allow appropriate, reliable, and valid measurement of project ownership (Hanauer et al., 2012; Hanauer and Dolan, 2014). The Project Ownership Survey (POS) consists of a set of validated scales that measure personal connection and importance of the research, the perceived social and scientific relevance of the project, and positive emotive responses to the course (Hanauer and Dolan, 2014). Based on both the initial qualitative data and the exploratory factor analysis, the final version of this survey was divided into two sections: items that come from the content analysis of project ownership (POS content items) and items that come from the emotion scales (POS emotion items). Both of these dimensions of the POS were used in analyzing the potential relationship of project ownership to networking. The relationship between project ownership and networking is potentially important, as it would suggest that a positive and engaging CURE experience results in the construction of supportive networks within and beyond the classroom that may have longer-lasting influences on career choices and persistence in the sciences. While this is not evaluated in this paper, establishing a relationship of this kind has significance for future projects.

The hypothesis that networking and project ownership should have a relationship was evaluated as part of the validation plan for the networking instrument. The basic assumption underpinning this analysis was that a sense of project ownership (involving degrees of excitement and personal engagement) should translate into the activity of talking to and interacting with others. The current regression analysis looked at the predicted value of project ownership scales for networking with parents, students, and professors. Together, this set of three studies should provide a comprehensive introduction to the properties and potential value of the networking scales.Figure 1 summarizes the overall design of this validation process.

Figure 1.

Schematic representation of the overall design of the validation procedure.

Participants

The participants in this study were all drawn from the Department of Biological Sciences at the University of Pittsburgh. For the factor analysis and regression analyses, 169 participants (N = 169), drawn from five different undergraduate laboratory courses, participated in this study. For the comparative study of two undergraduate laboratory courses on the networking scales, responses from a subset of 128 participants (n = 128) were used in this analysis. Enrollment in each of these courses was based on students’ personal choices in consultation with their academic advisors. There were no prerequisites or special requirements for either of the two courses that were compared, and students were not offered any incentive to participate in the survey. Demographic information relating to the participants is presented in Table 1. The request to participate in the survey and Web-based informed-consent process were conducted in accordance with Indiana University of Pennsylvania Internal Review Board (IRB) approval (log no. 14-302) and IRB approval from the University of Pittsburgh. The request to complete the survey was sent in the last two weeks of class, and all data were collected by the official end of the Fall semester 2014–2015.

Table 1.

Demographic characteristics of participants (n = 169)

Characteristic	n	%
Gender
Male	65	38
Female	103	61
Prefer not to respond	1	1
Class
First year	103	63
Sophomore	13	7
Junior	6	3
Senior	34	20
Prefer not to respond	13	7
Race/ethnic identification
White	113	67
Asian	21	12
Black or African American	7	4
Hispanic or Latino	2	1
Multiple ethnicities	11	7
Other	3	2
Prefer not to respond	12	7

Instrument Development

The networking tool was developed based on the theoretical principle that social networks are constructed and consist of groups of people in dialogue with one another (Lemke, 2001). The approach taken to develop the specific scales involved keeping the questions as direct and as simple as possible and basically asking whether students had discussed their scientific research with people in their personal and professional lives. The assumption was that knowing who students were speaking to concerning their scientific research should indicate the different social groups the student-researcher was constructing and the extent to which scientific knowledge might be shared across personal, educational, and professional networks.

The networking tool was designed to collect self-reported data on people to whom the participants spoke. In this sense, networking was defined in simple terms as a series of rating scales in which different discussion partners were defined, and participants specified whether these were people to whom they had spoken concerning their laboratory course research. The underpinning logic for this design was that networking is a communicative behavior in which information is shared. The networking tool aimed to capture this basic level of communicative behavior. The networking instrument is shown in Box 1.

Box 1.

Networking survey rating scales

Statement	Strongly disagree	Disagree	Neither agree nor disagree	Agree	Strongly agree
I have discussed my research in this course with my parents (or guardian).
I have discussed my research in this course with students who are not in my class but in my institution.
I have discussed my research in this course with my friends.
I have discussed my research in this course with professors other than my course instructor.
I have discussed my research with students who are not at my institution.

As the final stage of the scale development process, the networking scales were checked for their comprehensibility and validity with five undergraduate students actively involved in a CURE. Students were asked to read each item, use the rating scale, provide a verbal answer, explain how they understood the scale, and state whether the item was clear to them. Responses revealed that the items were clear, simple to understand, and could be rated. Importantly, the verbal responses to the items elicited narrative descriptions of the people to whom students were talking and the different considerations involved in these discussions. Students tended to differentiate between people who had a background in science and those who did not. More was said and specified when the respondent had a background and an interest in science. This final stage of instrument development established that the networking scales addressed and elicited information concerning participation and dialogue in different social networks.

Data-Collection Site

Data for this study were collected in the Department of Biological Sciences at the University of Pittsburgh. Students were drawn from a series of undergraduate laboratory courses offered at the university. For the comparative validation study of the networking scales, two different laboratory groups were specifically chosen. One laboratory group consisted of an established CURE and involved a freshman lab with a real research collaboration between multiple researchers, students, and institutions. This CURE offered undergraduate students the opportunity of a high-quality research experience built around the importance of authentic science. The course developed knowledge of bacteriophages and had a very accessible entry point, making it suitable for a wide range of student populations. The program provided exposure to current scientific knowledge, a microbiology wet lab, and bioinformatic procedures useful for this and subsequent courses. Built into the design of the program were a series of scientific networking and scientific communication opportunities, which included collaboration with other students and faculty at the University of Pittsburgh and at other universities across the United States, attending a yearly scientific symposium, and participating in Internet community activities. As a result of the extensive, interactive nature of this CURE, it was assumed that this lab course should elicit significant amounts of social networking. The second course was also a freshman laboratory. However, this second course was designed as a traditional laboratory experience, in which the focus was on learning a series of foundational wet lab procedures. Students were exposed to a series of controlled laboratory experiments with defined outcomes. The main aim of the course was for students to learn those laboratory procedures useful for the continuation of their studies. This lab did not involve any design features specifically directed at networking. The basic principle of the comparative design used in this study was that if the networking scales were valid, then they should be able to differentiate between the different levels of interaction, which were part of (or not part of) the design of these courses.

Procedure

Data collection was conducted through an online data-collection website (Qualtrics). An email request with a link to a data-collection instrument was sent directly to students in designated undergraduate laboratory courses in the Department of Biological Sciences at the University of Pittsburgh. The initial section of the survey involved agreement to participate in the study on an IRB-approved consent form. The data-collection tool included the 16 project ownership scales and the five networking scales. Data were collected during the last 2 wk of the Fall semester of the 2014–2015 academic year.

Data Analysis

As reported above, this paper involve three different studies of the networking scales. For the first study, which looked at the psychometric properties of the networking scales, conventional procedures of validation were used (Netemeyer et al., 2003). An exploratory factor analysis was used to explore the potential dimensionality of the scales, and then the internal consistency of the tool was evaluated by calculating Cronbach’s alpha. The second study considered the validity of the networking scales and used a known-groups approach. The data, which were initially screened for underpinning assumptions, were found not to be normally distributed. Accordingly, a nonparametric comparative approach was used. Mann-Whitney U-tests (a nonparametric t test) were calculated for each of the items on the networking scales to consider whether the scales systematically differentiated between the CURE laboratory with high expected networking and the traditional laboratory course with low expected networking potential. The third study involved a series of binary logistic regressions designed to consider whether project ownership scales had a predictive relationship with the construct of networking. As an initial stage, the networking scales of the following variables, “I have discussed my research in this course with my parents (or guardian),” “I have discussed my research in this course with professors other than my course instructor,” and “I have discussed my research in this course with students who are not in my class but in my institution,” were transformed from a five-point scale to a binary measure of “have discussed” or “have not discussed” their research. The first three points on the rating scales (strongly disagree, disagree, and neither disagree nor agree) were categorized as “has not discussed their research”; the last two points on the scale (agree and strongly agree) were categorized as “has discussed their research.” Following the transformation of the three networking scales, the project ownership scales were used as a predictive model in a binary logistic regression analysis. This logistic regression allowed the determination of the impact of multiple independent predictor variables, presented simultaneously, to predict membership in the category of participants having or not having discussed their research with different specified parties.

RESULTS

Psychometric Properties of the Networking Scales

Dimensionality.

Because the networking scales represent a new instrument, an exploratory factor analysis was conducted to establish the internal structure and the dimensions of the networking construct (Thompson, 2004). As reported above, 169 participants completed the networking scales, with a participant to variable ratio of 33:1. As a first stage, a Kaiser-Meyer-Olkin measure of sampling adequacy was calculated with the result of 0.827. This result, which is well above the 0.5 benchmark, indicates an adequate sample and, accordingly, a full exploratory factor analysis was conducted. Descriptive statistics for each of the networking items to be used in the factor analysis were calculated to make sure that the assumption of multivariate normality was not violated. Bartlett’s test indicated that the data were suitable for an exploratory factor analysis (χ² [10] = 472.28, p < 0.0001). A maximum-likelihood exploratory factor analysis was conducted to determine the internal structure of the survey and the dimensions of project ownership.

For evaluating the number of factors involved in the analysis, a trianalytical approach was used. This included the Kaiser criterion of keeping only factors with eigenvalues greater than one, graphing and visual analysis of the scree plot of eigenvalues, and parallel analysis comparison of factor eigenvalues with Monte Carlo procedure randomly generated eigenvalues. Initially, a scree plot of eigenvalues was graphed. As can be seen in Figure 2, the scree plot indicates a one-factor solution. Consideration of the eigenvalues of the factors using the Kaiser criterion also suggests that only the first factor should be included. The Monte Carlo procedure functions as a null hypothesis and provides threshold levels for eigenvalues that can be used above chance at the 0.05 significance level. As can be seen in Table 2, comparison of eigenvalues from current study data and the Monte Carlo simulation revealed that only the first factor was above the chance level, suggesting a one-factor solution. Consideration of the percentage of variance explained revealed that the first factor, by itself, accounted for 68.2% of the variance, which is above the 50% threshold of variance suggested as an acceptable outcome for a factor analysis solution. All three analyses concur on a single-factor solution.

Figure 2.

Scree plot of eigenvalues.

Table 2.

Eigenvalues, percentage of variance explained, and Monte Carlo simulation eigenvalues

Factor	Eigenvalue	% Variance explained	Monte Carlo simulation eigenvalue
1	3.41	68.2	1.22
2	0.63	12.6	1.09
3	0.47	9.4	0.99
4	0.3	6.1	0.90

Using a maximum-likelihood exploratory factor analysis, a one-factor solution was extracted. Table 2 presents the factor loadings for the individual networking items. As can be seen in Table 3, all factor loadings are high, ranging between 0.57 and 0.90. The results of the exploratory factor analysis indicate that the networking items form a single underpinning construct with all items having an important function.

Table 3.

Factor loadings for networking scales (n = 169)

Networking item	Factor loadings
I have discussed my research in this course with my parents (or guardian).	0.9
I have discussed my research in this course with professors other than my course instructor.	0.84
I have discussed my research in this course with students who are not in my class but are in my institution.	0.77
I have discussed my research in this course with my friends.	0.75
I have discussed my research with students who are not at my institution.	0.57

Reliability.

The internal consistency of the networking instrument was calculated using Cronbach’s alpha with the result α = 0.88, which indicates high levels of consistency for the tool. To further understand the internal consistency of the tool, we calculated item-total correlations for each item. This analysis consists of correlating each item with the sum of the items (total score) and allows the identification of items that might reduce reliability (Guilford, 1953). In the current analysis, deletion of any of the items would have reduced the alpha level. Accordingly, based on the results of the reliability analysis, no items were chosen for deletion from the scale, and the whole instrument is to be considered highly reliable.

Differentiating Abilities of the Networking Scales

As reported above in the Methods section, the aim of this second study was to explore whether the networking scales could differentiate between a laboratory course with low and high expected networking potential. Figure 3 presents the comparison of means between the two groups. Table 4 summarizes the means, SDs, medians, and pairwise Mann-Whitney U-test comparison of the two laboratory groups. As can be seen in Table 4, all the networking scale ratings for participants in the CURE laboratory are significantly higher than those of the traditional laboratory participants. As seen in Figure 3, the difference in means between the two groups ranges from 1.8 to 1.04 points on a five-point scale. The difference in medians between the two groups ranges from 3 to 1 on a five-point scale. All scales are significantly different on the Mann-Whitney U-test at the 0.0001 level. The results of this comparative validation study suggest that the networking scales can differentiate between situations of low and high networking in course design.

Figure 3.

Means and 95% confidence intervals (error bars) for two groups on networking items.

Table 4.

Means, SDs, Medians, and Mann-Whitney U-test for networking categories and two laboratory courses (n = 128)

Category	CURE (n = 44)	Traditional lab (n = 84)	Mann-Whitney U-test
I have discussed my research in this course with my parents (or guardian).	4.36	2.5	471*
	(0.81)	(1.28)
	5	2
I have discussed my research in this course with professors other than my course instructor.	2.91	1.87	992.5*
	(1.3)	(0.99)
	3	2
I have discussed my research in this course with students who are not in my class but are in my institution.	4.16	2.89	785*
	(0.88)	(1.23)
	4	3
I have discussed my research in this course with my friends.	4.27	2.92	705*
	(0.95)	(1.24)
	4	3
I have discussed my research with students who are not at my institution.	3.7	2.35	806*
	(1.25)	(1.2)
	4	2

*Asymp. Sig. p < 0.0001.

Regression Analyses

The final validation study conducted on the networking scales consisted of considering the relationship between these scales and the psychosocial measure of project ownership. A series of logistic regression analyses were conducted to predict whether students had or had not discussed their research with their parents, professors, and other students, using rating scales for project ownership content and project ownership emotion scales as predictors. Table 5 presents the findings of each of the linear regression models. As can be seen in Table 5, all the construct models were found to be statistically significant when tested against a constant-only model (see chi-square significance). This indicates that the project ownership scales as predictors reliably distinguished between students having or not having discussed their research with parents, professors, and other students at their institution (but not in their class).

Table 5.

Binary logistic regression analysis for project ownership content and emotion scales to predict discussion with parents, professors and other students

Construct	Chi-square and Sig.	Nagelkerke’s R2	Hosmer and Lemeshow test Sig.	% of correct classification with modela	Significant scales	Exp(B)
“I have discussed my research in this course with my parents (or guardian).”
Project ownership content	53.73	0.37	0.29	75.4	POSexcitingb	1.88
	p < 0.001			(53.3)	POSpersonal achievement	1.84
					POSsolve world problem	1.69
Project cwnership emotion	49.47	0.35	0.06	74.5	POSamazed	1.73
	p < 0.001			(53.3)	POShappy	1.49
“I have discussed my research in this course with professors other than my course instructor.”
Project ownership content	46.01	0.38	0.19	85	POSsolve world problem	1.89
	p < 0.001			(80.2)	POSresearch question important	1.79
					POSresponsible	1.59
Project ownership emotion	42.06	0.36	0.67	86.7	POSdelighted	1.48
	p < 0.001			(80)	POSjoyful	1.47
“I have discussed my research in this course with students who are not in my class but in my institution.”
Project ownership content	57.38	0.39	0.71	74.9	POSresponsible	1.67
	p <0.001			(58.1)	POSexciting	1.47
					POSfaced challenges	1.41
Project ownership emotion	35.07	0.26	0.37	71.5	POSdelighted	1.6
	p < 0.001			(57.6)
aThe percentage of correct classification without the model variables is shown in parentheses.
bThe project ownership survey items and identifiers are as follows:

Project ownership content scales					Scale identifier
My research will help to solve a problem in the world.					POSsolve world problems
My findings were important to the scientific community.					POSscientific community
I faced challenges that I managed to overcome in completing my research project.					POSfaced challenges
I was responsible for the outcomes of my research.					POSresponsible
The findings of my research project gave me a sense of personal achievement.					POSpersonal achievement
I had a personal reason for choosing the research project I worked on.					POSpersonal project
The research question I worked on was important to me.					POSresearch question important
In conducting my research project, I actively sought advice and assistance.
My research project was exciting.					POSexciting
My research project was interesting.					POSinteresting

Project ownership emotion scales
To what extent does the word delighted describe your experience of the laboratory course?					POSdelighted
To what extent does the word happy describe your experience of the laboratory course?					POShappy
To what extent does the word joyful describe your experience of the laboratory course?					POSjoyful
To what extent does the word amazed describe your experience of the laboratory course?					POSamazed
To what extent does the word surprised describe your experience of the laboratory course					POSsurprised
To what extent does the word astonished describe your experience of the laboratory course?					POSastonished

However, the percentage of variation in the outcome variable and proportions of correct classification differs between the models. As a rule of thumb, you would want close to 30% of the variation to be explained by a proposed model and a correct classification of at least 65% of cases. In addition, you would want to see added value in the percentage of correct classification over a model that classifies the cases without the variables. Based on these criteria, the scales for project ownership, content, and emotion predict future networking with parents, professors, and students. In relation to the networking scale of discussing research with parents, the project ownership content model has a Nagelkerke’s R² of 0.37, which indicates a moderate relationship between predictors and grouping, and this prediction model has a 75.4% success rate, which is an increase of 22.1% over the constant without variables model prediction. For project ownership emotion scales, on the same networking scale, the model has a Nagelkerke’s R² of 0.35, which indicates a moderate relationship between predictors and grouping, and this prediction model has a 74.5% success rate, which is an increase of 21.2% over the constant without variable model prediction. Analysis of the logistic coefficients Exp (B) for the specific items in the project ownership construct reveals that the following items—“My research project was exciting,” “The findings of my research gave me a sense of personal achievement,” “My research will help to solve a problem in the world,” “‘Amazed’ describes my experience in this course,” and “‘Happy’ describes my experience in this course”—made the most significant contribution to the prediction. Each of these variables was significant at the 0.01 level using the Wald criterion. Importantly, the sense that the “research project was exciting” and an increased “sense of personal achievement,” when increased by one unit, raised the odds of students discussing their research with their parents by 1.8 times, suggesting a particularly significant role for these particular items. Overall, the regression analysis of the project ownership model suggests a predictive relationship between the presence of project ownership within a student’s laboratory research experience and the later discussion of the research with parents.

In relation to the networking scale of discussing participants’ research with professors other than their course instructors, the project ownership content model has a Nagelkerke’s R² of 0.38, which indicates a moderate relationship between predictors and grouping. This prediction model has an 85% success rate, which is an increase of 4.8% over the constant without variables model prediction. For project ownership emotion scales, on the same networking scale, the model has a Nagelkerke’s R² of 0.36, which indicates a moderate relationship between predictors and grouping. This prediction model has an 86.7% success rate, which is an increase of 6.7% over the constant without variables model prediction. Analysis of the logistic coefficients Exp (B) for the specific items in the project ownership construct reveals that the following items—“My research will help to solve a problem in the world,” “The research question I worked on was important to me,” “I was responsible for the outcomes of my research,” “‘Delighted’ describes my experience in this course,” and “‘Joyful’ describes my experience in this course”—made the most significant contributions to the prediction. Each of these variables was significant at the 0.01 level using the Wald criterion. Importantly, the scale “My research will help to solve a problem in the world,” when increased by one unit, raised the odds of students discussing their research with professors other than their course instructors by 1.8 times. Overall, the regression analysis of the project ownership model suggests a predictive relationship between the presence of project ownership and the discussion of their research with professors who are not the classroom instructors.

In relation to the networking scale of discussing participants’ research with students who are not in the participants’ class but at their school, the project ownership content model has a Nagelkerke’s R² of 0.39, and this indicates a moderate relationship between predictors and grouping. This prediction model has a 74.9% success rate, which is an increase of 16.1% over the constant without variables model prediction. For project ownership emotion scales, on the same networking scale, the model has a Nagelkerke’s R² of 0.26. This is a weak relationship between predictors and grouping and below the 30% variance explained threshold. Accordingly, the project ownership emotion scale model does not seem to be a good predictor of group membership. Analysis of the logistic coefficients Exp (B) for the specific items in the project ownership content construct reveals that the items “I was responsible for the outcomes of my research,” “My research was exciting,” and “I faced challenges that I managed to overcome in completing my research project” made the most significant contributions to the prediction. Each of these variables was significant at the 0.01 level using the Wald criterion. Overall, the regression analysis of the project ownership content model suggests a predictive relationship between the presence of project ownership within a student’s laboratory research experience and the later discussion of the research with students who are not in the class but at the same institution. The project ownership emotion model was not found to be predictive of future discussion with other students.

DISCUSSION

This paper presented a new instrument dealing with the degree to which students in research experiences discuss and network with other people about their research. The networking instrument covers personal and professional social networks and, as shown in the data presented here, is both valid and reliable. The factor analysis showed this instrument to consist of a single factor. Reliability was analyzed and returned a Cronbach’s alpha value of 0.88, signifying high internal consistency. A comparison of two undergraduate research experiences—one a CURE that directed resources at developing professional networks and the other a traditional lab course—demonstrated that the networking scales differentiated between the two situations, offering added validity to the use of the tool in exploring outcomes of course design. Finally, the relationship between networking and project ownership was evaluated. The project ownership content scales were related to networking with parents, professors, and other students. The project ownership emotion scales were related to networking with parents and professors but only weakly to networking with other students. Overall, the data from the three studies presented here indicate that the networking scales have internal consistency, consist of a unitary dimension, can differentiate between research experiences with low and high networking designs, and are related to project ownership scales. In this sense, the current study validates the networking survey and facilitates its usage in other studies by researchers for evaluation of student outcomes purposes.

While this paper is primarily aimed at providing validation for a new variable to be considered in the analysis of the student outcomes while participating in a CURE, it is important to consider the relationship between the networking construct as measured by the new networking scales and current issues of concern to science educators. As specified in the introduction to this paper, retention in the sciences is a major concern at the federal, state, institutional, and disciplinary levels. It is interesting to note that the three major models of achieving retention in the sciences all specify the importance of students becoming members of the scientific community (Estrada et al., 2011; Graham et al., 2013; Corwin et al., 2015). Corwin et al. (2015), utlizing a situated-learning theory of learning in discussing CUREs, considered continued interaction with a more expert member of the scientific community an important aspect of retention (Lave and Wegner, 1991). Estrada et al. (2011), primarily interested in exploring retention of minority students in the sciences, explained retention in terms of levels of social integration and aspects of social influence. Finally, Graham et al. (2013), in their persistence model, which is organized around the enhancement of student motivation, specified the importance of joining a science, technology, engineering, and mathematics learning community in achieving retention. Inherent in all three of these models of retention is the idea that becoming and staying a member of a scientific community promoted retention of students (including members of minority groups). The networking scales provide a basic measure of membership in a community. If you are talking to other students and professors about your research, you are de facto in a community relationship. Because the scales ask about discussing “your research” with these other discussion partners, the scales measure depth of community membership in relation to the different cycles. Although the analysis was not conducted here, it can be hypothesized that measures, such as the internalization of scientific community values (Chemers et al., 2011; Estrada et al., 2011), are mediated through professor networking scales and that scientific identity is related to the degree to which you talk to your parents and other students about your research.

A second area of concern to scientists, science educators, funding agencies, policy makers, and educational institutions is the degree to which the broader public achieves scientific literacy (Gormally et al., 2012). Scientific literacy covers a wide range of positions, from quite initimate, personal experience and internalization of scientific community conventions, values, and abilities, to a limited understanding of but positive attitude toward the generation of scientific knoweldge. Underpinning the urgency to develop scientific literacy within the public is the fact that many issues faced by society require the ability to evaluate scientific evidence and understand basic issues in science, such as the difference between a virus and a bacteria, in relation to outbreaks of infectious diseases. Networking with students learning in CUREs may have a role to play in this development of scientific literacy. Once again, on a very basic level, if you talk with friends and close family about your research conducted in laboratory courses, some degree of familiarity with the science should emerge, even if this is partial and happens through a secondary source. In this sense, the degree of conversation beyond the classroom and in the realm of personal social networks raises interesting options for extending classroom learning into the community. The potential here is that CUREs could, through societal networking, develop scientific literacy for people who are not phsyically in the classroom but are related or connected to a young student doing research in a laboratory course. While the data for this hypothesis have not yet been collected, a commonsense position suggests that if you are talking to people about your research, then there is the possibility that they are learning something about science. The networking survey can assess one component of this enhanced impact of a CURE.

On a more basic level, it is worth considering in conjunction two of the specific results found in the current validation study of the networking scales. A close consideration of the specific project ownership scales that contributed to the relationship with networking scales reveals some important aspects of the students’ responses to their laboratory courses. In relation to increased discussion with parents, personal excitement with the course, a sense of personal achievement, and the feeling that your research will solve a real-world problem were found to make an important contribution. Simply put, if the CURE you are learning in is exciting, gives you a sense of personal achievement, and is real, then you talk about it with your family. In relation to increased discussion with other professors, a sense that the research question has personal importance and that your research will solve a real-world problem contributed to the regression model. If you feel a personal connection to the question you are exploring, and you understand the societal significance of this work, you will seek out and talk to other scientists who can perhaps contribute to the development of your work or benefit from hearing what you are doing. We already know from the second study presented here that the networking scales significantly differentiate between types of laboratory course. As such, the findings presented here demonstrate a broader differentiation between laboratory course designs. Some types of courses (CUREs, for example) may have deep connections within broader society, whereas other designs (such as the traditional, procedure-based laboratory) do not foster any social or professional networking.

As with all studies, the research presented here raises more questions than it answers. It is important to consider additional relations between established psychosocial variables and networking and to collect data beyond the single setting used in this study. In particular, it seems important to explore networking with those groups of students who traditionally have found it difficult to enter into scientific communities. Networking for minority students may be especially interesting if relations between particular course designs and systematically enhanced or decreased networking is found, whether this is in the personal or professional realms. In any case, the current study provides validation for a tool that can collect data on the self-reported networking outcomes of particular laboratory course experiences. This should add a variable that can be measured and be evaluated when we consider how CUREs function and their value in terms of student outcomes.

The aim of the paper presented here was to introduce, develop, and validate a tool for measuring the self-reporting of networking in the personal and professional societal realms. The validation study provides information on the usage of the tool and its potential as part of the repertoire of assessment options that can be applied to evaluate different undergraduate laboratory courses and CUREs. On its simplest level, the networking survey collects data on whether students talk about their research and to whom and provides some insight into the communities they are constructing while participating in laboratory course. Future research may establish that networking is an important component of retention, valuable in differentiating course experiences, and a mechanism for evaluating the broader societal impacts of CUREs.