Incorporating an Interactive Statistics Workshop into an Introductory Biology Course-Based Undergraduate Research Experience (CURE) Enhances Students’ Statistical Reasoning and Quantitative Literacy Skills

Abstract

Course-based undergraduate research experiences (CUREs) provide an avenue for student participation in authentic scientific opportunities. Within the context of such coursework, students are often expected to collect, analyze, and evaluate data obtained from their own investigations. Yet, limited research has been conducted that examines mechanisms for supporting students in these endeavors. In this article, we discuss the development and evaluation of an interactive statistics workshop that was expressly designed to provide students with an open platform for graduate teaching assistant (GTA)-mentored data processing, statistical testing, and synthesis of their own research findings. Mixed methods analyses of pre/post-intervention survey data indicated a statistically significant increase in students’ reasoning and quantitative literacy abilities in the domain, as well as enhancement of student self-reported confidence in and knowledge of the application of various statistical metrics to real-world contexts. Collectively, these data reify an important role for scaffolded instruction in statistics in preparing emergent scientists to be data-savvy researchers in a globally expansive STEM workforce.

INTRODUCTION

Course-based undergraduate research experiences (CUREs) in the biological sciences have provided a unique platform for immersing students in the process of authentic scientific discovery (1, 2). Inherent in these experiential opportunities is the expectation that students will develop questions and hypotheses, identify methods to address those hypotheses, and analyze and communicate the outcomes of their investigations (1–3). While current research indicates that students are increasingly adept at “thinking like a scientist” as a result of participating in CUREs (4, 5), similar studies reveal that students often exhibit low self-efficacy in terms of quantitative reasoning and literacy in such contexts and in the domain more broadly (6–8). Furthermore, although substantial efforts have been made to enhance student proficiency in these areas (e.g., 9–12), such efforts are often: a) designed to address a single laboratory-based research question with a finite set of potential, parallel student-generated hypotheses; b) time-intensive; c) conducted in course contexts outside of the CURE learning environment; d) focused on instructor-generated datasets; and/or e) evaluated almost exclusively using student self-reported data.

To address these concerns, we developed a one-hour, interactive statistics workshop for use in the introductory cell and molecular biology CURE at the University of Northern Colorado (8). The principal learning goal of this workshop was to engage students in analyzing their own data, in real-time, in preparation for a final oral presentation and written laboratory report in the course. In order to determine the efficacy of the workshop, mixed-methods analyses were used to explore the following central questions:

1. What impact does the interactive workshop have on students’ development of statistical reasoning and quantitative literacy skills within a biological sciences context?

2. What misconceptions do students possess about statistical reasoning prior to and following the workshop?

3. How does engagement in the workshop influence students’ self-reported attitudes toward and knowledge of statistics?

We hypothesized that participation in the workshop would enhance students’ statistical abilities, including their quantitative literacy, given the explicit focus of the workshop on these outcomes (see Supplemental Materials and the Interactive Statistics Workshop: A Brief Description section below). In addition, because the majority of students enrolled in the CURE were freshmen and had not received prior instruction in statistics (per course demographic information), we predicted that contextualizing data analysis within the parameters of their own research projects would motivate students and enhance their positive affect toward the application of statistics in the biological sciences. These hypotheses are supported by prior literature, which demonstrates that incorporation of data analytics and quantitative methods into science curricula positively impacts student comprehension of and affect toward statistics in the science classroom (e.g., 11, 13, 14).

INTERACTIVE STATISTICS WORKSHOP: A BRIEF DESCRIPTION

Recent reform efforts at the University of Northern Colorado have led to the adoption of CUREs across the entire first-year biological sciences sequence. Development and implementation of the interactive statistics workshop within the first course in that sequence (introductory cell and molecular biology) was a direct response to previously observed CURE outcomes, which revealed a lack of student self-efficacy with respect to performing quantitative analyses and communicating the resultant findings (8). Retrospective analysis of students’ final laboratory reports (data not shown) further revealed inappropriate use of statistical tests (e.g., use of multiple independent t-tests rather than an ANOVA procedure) and a substantial lack of data interpretation. Consequently, the workshop was designed to: 1) increase increase students’ statistical reasoning skills; 2) enhance students’ quantitative literacy; and 3) promote positive affect toward statistics and its applications within biological contexts. In an effort to be both cost- and resource-efficient, workshop duration was established as one hour, and only standard materials (e.g., video, worksheet exercises) were employed. An expanded description of the workshop as well as exemplar student research questions can be found in the Supplemental Materials (Appendix 1).

METHODS

Participant recruitment

Participants (n = 80) represented a convenience sample consisting of all students enrolled in four sections of an introductory cell and molecular biology CURE at a midsize, doctoral degree-granting institution in the spring 2016 semester. Participants were predominantly first-semester freshmen majoring in the science, technology, engineering, and mathematics (STEM) disciplines (~59%) (see Appendix 2 for complete demographic data). In an effort to reduce bias due to repeated exposure to course content, only those individuals completing the course for the first time were included in our analyses. Participants completed a three-hour, structured laboratory exercise on statistical methods in the sixth week of the semester prior to participating in the interactive workshop (week 14); no additional formal statistics instruction was provided.

Measurement of statistics comprehension

In order to evaluate the extent to which the interactive workshop mediated students’ development of statistical reasoning skills in the context described, the Statistical Reasoning in Biology Concept Inventory (SRBCI) (15) was administered in pre/post-intervention format. The SRBCI consists of 12 multiple-choice items designed both to probe undergraduate students’ understanding of statistics within a biological framework and to identify common misconceptions related to statistical reasoning (e.g., incomplete comprehension of the purpose of hypothesis testing). The assessment is delivered as a 17-minute, timed PowerPoint presentation with an accompanying handout containing supplemental figures associated with select questions (see (15) for specific instructions). Students are required to respond to each question before the designated time expires (the time allotted varies per question, although a 10-second warning is presented on each slide to cue students that the subsequent question is about to be delivered) and are instructed to record all responses on a separate answer sheet created and provided by the course instructor. With regard to the present study, each item was first scored as either correct (‘1’) or incorrect (‘0’) and entered into SPSS (v. 23, IBM). A paired t-test approach was subsequently used to assess for shifts in student performance over the course of the intervention, and average individual learning gains were tabulated in accordance with Hake (16). In addition, the types and relative frequencies of student misconceptions on each SRBCI item were determined (see (15) for a complete list of misconceptions associated with each assessment question; Appendix 3) and entered into SPSS (v. 23, IBM). Paired t-test analyses with Bonferroni correction were employed to ascertain the prevalence of these misconceptions relative to other, alternate conceptions, as well as the correct response for each item, on both the pre- and post-intervention SRBCI diagnostics.

Measurement of quantitative literacy

A modified version of the Association of American Colleges and Universities’ (AAC&U) Quantitative Literacy VALUE rubric (https://www.aacu.org/value/rubrics/quantitative-literacy) (Appendix 4) was used to assess participants’ (n_groups = 9, representing 50% of the total number of student groups) quantitative writing proficiency on a variety of subscales (e.g., representation and interpretation of data), as evidenced in a random sample of end-of-semester laboratory reports associated with the course. Reports were blinded and scored independently by two researchers with expertise in the biological sciences and bioeducation disciplines. High inter-rater reliability was observed (κ = 0.88, p < 0.001), with all disputes resolved through conversation between the two coders until resolution was achieved. Separate composite scores were tabulated for those rubric subscales addressed in the intervention (interpretation, calculation, application/analysis) and those that were not (representation, assumptions, communication), with a maximum score of three points possible for each unique subscale. These scores were then compared using the Wilcoxon signed-rank test to examine the extent to which variation existed in participants’ quantitative literacy skills as a function of workshop content.

Student perceptions of learning gains (SPLG)

In addition to the cognitive data referenced above, participants were asked to complete a brief survey immediately prior to and following the intervention, the intent of which was to examine changes in their perceived confidence in comprehending and applying basic statistical principles as a result of participation in the workshop. This survey consisted of seven, Likert-item questions (Appendix 5) adapted from Marsan et al. (12) and based upon structured conversations with faculty and graduate teaching assistants (GTAs) at the university at which this research occurred (Olimpo, unpublished). The post-intervention survey contained one additional open-ended question (What elements of the workshop did you find most helpful/beneficial to your understanding of the topic of statistics and data reporting, and why?) designed to elicit feedback regarding students’ beliefs about the efficacy of the intervention. Open-ended responses were analyzed using a descriptive interpretive approach (17), with emergent themes identified following iterative cycles of open and axial coding. Each student response was coded independently by two researchers with expertise in the fields of biological sciences and bioeducation. High inter-rater reliability was observed between coders (κ = 0.95, p < 0.001), with all disputes resolved by a third researcher with similar expertise. Descriptive statistics were tabulated for Likert-item questions, with potential rankings ranging from “1” (strongly disagree) to “5” (strongly agree) and the data entered directly into SPSS (v. 23, IBM) for future analysis.

RESULTS

To reduce potential bias introduced as a result of variability in laboratory GTA instructors, pre- and post-intervention SRBCI scores were first analyzed using a multivariate analysis of variance (MANOVA) procedure. Results indicated no significant difference in outcomes based on GTA instructor (F(4,152) = 1.085, p = 0.367, Wilk’s Λ = 0.945, η_p² = 0.028). Furthermore, while we acknowledge that pedagogical strategies employed in the classroom likely varied between instructors, it is important to note that all workshop materials (e.g., videos; PowerPoints; worksheets) were identical between sections, and a standardized GTA agenda for the workshop was distributed to all instructors (see Supplemental Materials). Given the observed variability in student demographic characteristics (in particular, prior statistics coursework and academic major) (Appendix 2), we likewise examined the potential impact of these factors on student performance and affective metrics, as detailed in the Methods section. Results of a MANOVA procedure indicated no statistically significant main effect for either major (F(8,61) = 0.685, p = 0.703, Wilk’s Λ = 0.918, η_p² = 0.082) or completion of prior statistics coursework (F(8,61) = 0.708, p = 0.683, Wilk’s Λ = 0.708, η_p² = 0.085). Remaining assumptions for performing parametric and non-parametric tests were confirmed prior to data analysis.

Participation in the interactive workshop promotes students’ development of statistical content knowledge

In order to test our hypothesis that participation in the interactive workshop enhanced students’ development of statistical reasoning skills, participants’ SRBCI responses prior to and following the session were compared using a paired t-test procedure. This analysis revealed a statistically significant pre-intervention (M = 31.14%; SD = 14.63%) to post-intervention (M = 36.88%; SD = 16.65%) increase in performance (t(79) = −3.505; p = 0.001; Cohen’s d = 0.366). Average individual learning gains were 7.17% (SD = 21.77%).

Unpacking students’ misconceptions regarding statistics

As a mechanism to further explore student performance on the pre- and post-intervention SRBCI assessments, a series of paired t-tests with Bonferroni correction were conducted to examine the types and frequencies of misconceptions evidenced by each participant following stratification by questionnaire item number and time of diagnostic administration (preceding or succeeding the intervention). Collectively, these data revealed that students possessed alternate conceptions regarding the inferential value of 95% confidence intervals (CIs) prior to participation in the workshop, including the belief that 95% CIs do not provide information about statistical significance (e.g., 74% of responses on item #1; 79% of responses on item #3) and that, as long as average scores between comparison groups are different (regardless of 95% CI), there is a significant effect present (e.g., 33% of responses on item #12) (Appendix 6). Furthermore, analyses indicated that a moderate percentage of students experienced difficulties in interpreting information pertaining to sample size, perceiving, for instance, that altering the sample size would not have an effect on the limits of 95% CIs (e.g., 41% of responses on item #11) or that adjusting the sample size between groups would necessarily result in a decreased chance to observe significant differences between those groups (e.g., 39% of responses on item #11).

Examination of post-intervention SRBCI data offers an informative counterpoint regarding the “evolution” of these misconceptions over time. Students’ perceptions that 95% CIs do not provide information about statistical significance decreased on select items (e.g., a reduction in response rate from 79% to 68% on item #3) as did their belief that 95% CIs must be non-overlapping when conducting comparisons between all groups in order for a significant effect to be observed (e.g., a reduction in response rate from 40% to 29% on item #7). However, the presence of misconceptions related to the influence of sample size on experimental outcomes increased at times (for comparative purposes, see item #11), and the notion that combining data sets with slight variations in experimental design would not introduce bias into one’s study persisted following the intervention, as examples (Appendix 6). While our intent was not to perform a direct, comprehensive comparison of the change in frequency of each misconception for each assessment item over the duration of the intervention (in fact, multiple SRBCI items assess the same or similar misconceptions), Appendix 6 provides a representative account of the broader shifts in students’ statistical reasoning abilities as they pertain to implementation of the interactive workshop.

Assessment of students’ quantitative literacy

Given the learning objectives of both the Tigriopus CURE (8) and the interactive workshop, we found it imperative to further investigate students’ level of quantitative literacy (as defined in https://www.aacu.org/value/rubrics/quantitative-literacy) through examination of a random sample of end-of-semester student laboratory reports associated with the course (n_reports = 9). Qualitative content analyses (18) revealed anticipated differences in performance between research teams, most notably on those dimensions related to statistical calculations as well as assumptions pertaining to data analysis and reporting (data not shown). Because the focus of the workshop was primarily related to the calculation, interpretation, and application of quantitative analyses of student-generated datasets, an a priori decision was likewise made to conduct a non-parametric comparison between participants’ (n_groups = 9) average, aggregated score on these three elements and their average, aggregated score on the three dimensions not explicitly addressed during the intervention (representation, assumptions, communication). The Wilcoxon signed-rank test demonstrated a statistically significant difference in composite scores between those factors addressed in the workshop (M = 6.22; SD = 1.48) and those that were not (M = 4.78; SD = 1.30) (Z = −2.958; p = 0.009) (Fig. 1).

FIGURE 1.

Measurement of various dimensions of students’ quantitative literacy, as evidenced on final written laboratory reports. Data indicate a statistically significant difference in performance between those concepts discussed in the workshop and those that were not. *p = 0.009.

While these data suggest a potential positive association between participation in the intervention and students’ development of related quantitative literacy skills, we wish to acknowledge that outcomes must be interpreted with caution given the absence of an equivalent pre-intervention written report.

Students’ perceived confidence in understanding and applying statistical concepts is positively impacted as a result of participating in the intervention

Participants’ Likert-item responses on the pre- and post-intervention SPLG survey were compared using a series of paired t-tests with Bonferroni correction. These analyses revealed a statistically significant increase in self-reported confidence on the majority of assessment items (p < 0.001 for indicated significant comparisons; Fig. 2). Perceived increases in students’ ability to identify appropriate statistical methods with which to analyze their data, interpret p-values and statistical tests associated with such analyses, and explain relevant statistical parameters (e.g., alpha values and error bars) were of notable interest. Effect sizes (not shown) indicated that these factors exhibited the largest increases in confidence over the duration of the intervention, yielding insight into the success of the interactive workshop in promoting student affect in the context described.

FIGURE 2.

Student perceptions of learning gains (SPLG) associated with participation in the interactive statistics workshop. ^ap < 0.001.

Students’ perceptions of the interactive workshop

In addition to assessing the impact of the interactive workshop on cognitive and non-cognitive student outcomes in the domain, we endeavored to understand what features of the intervention had the potential to mediate those observations. Student responses to the open-ended prompt on the end-of-term SPLG survey were analyzed using a descriptive interpretive approach (17), revealing five emergent themes within the dataset: 1) provision of content knowledge (e.g., definition of an independent t-test); 2) application of content/statistical methods to student experimentation; 3) enhancement of mathematical/statistical skills; 4) connection to real-world scenarios or examples; and 5) interaction with knowledgeable others (e.g., GTAs). As a consequence of iterative analysis of student excerpts, it was likewise determined that the first theme could be further disaggregated into two sub-themes, namely: “knowledge of specific statistical procedures” and “knowledge of whether to select a graph or table to depict one’s data.”

Among the aforementioned categories, provision of content knowledge was cited most frequently (67.5% of participants), with 45.5% of individuals stating that the interactive workshop was beneficial in enhancing their understanding of statistical procedures (Table 1). Approximately one-quarter of participants emphasized the importance of the workshop in providing them with dedicated time to apply session content to their own research experiments in the course (e.g., to analyze student-generated data), and ~15% noted that the experience had improved their ability to perform mathematical and/or statistical skills (e.g., how to perform relevant calculations). To a lesser, albeit important, extent, incorporation of real-world examples and scenarios into the workshop (6.5% of responses) as well as opportunities to interact with knowledgeable others (e.g., GTA; 7.8% of responses) were also found to be of value to students. When considering these data as a collective, it is important to note that more than 25% of participant responses were identified as belonging to two or more of the above themes. One student noted, for instance, that “the workshop provided knowledge on concepts of statistical analysis and made it possible to use our own experiment and data results to find these statistics. This will make it much easier to transfer this information into our [final research] paper.” In this sense, results indicate that the effectiveness of the intervention is predicated upon the interrelationships between multiple factors, including those of both an epistemic and contextual nature. When considered in conjunction with one another, rather than in isolation, these factors have the potential to positively influence student learning in CURE contexts that incorporate statistical approaches within the curriculum.

TABLE 1.

Student responses to the question “What elements of the workshop did you find most helpful/beneficial to your understanding of the topic of statistics and data reporting, and why?”

Theme	Student Responses within Themea	Sample Student Responses
Provision of Content Knowledge	68%	“[The workshop] provided a better explanation of when to use the t-test vs. ANOVA, as well as how to interpret our results based on the [alpha] significance level.” “The video helped me clarify what terms meant that I had forgotten.” “I liked how it went step-by-step that I could follow, so I learned it while going through it, and the background of the numbers… I got what they mean.”
Sub-Theme: Knowledge on Specific Statistical Tests	45%
Sub-Theme: Knowledge on Graphs/Tables	8%
Application of Content to Students’ Research	23%	“The workshop helped me understand how to relate it [session concepts] to our team’s hypothesis.” “Applying the principles to our experiment helped us further analyze our data.” “The physical analysis of our data made it a lot easier to understand the statistics.”
Perform Mathematics/Statistical Skills	15%	“I [learned] how to calculate and interpret a p-value.” “Learning how to do an ANOVA [was most helpful], as in not actually just putting in the data, but actually running it and producing results.”
Incorporation of Real-World Scenarios	6%	“I found that the instructional video was the most helpful because it gave us examples on how to work problems and how to achieve our answers.” “The example situations with the statistics were the most helpful because it made it easier to understand how to perform different statistical tests.”
Interaction with a Knowledgeable Other	8%	“It was helpful to just be able to use our TA for the things we were struggling with.” “The [workshop] gave us time to work with our TA and ask him questions.” “I didn’t have to figure it out by myself. I had the teacher there to help me.”

^an = 80; student responses were coded into multiple categories, as appropriate.

CONCLUSIONS / IMPLICATIONS

The continued advent of novel CUREs has resulted in a marked increase in the need for students to possess quantitative literacy and data analytic skills vital for interpretation and application of research outcomes (e.g., 19). While prior modules exist to assist students in acquiring proficiency in these areas (10, 11, 20), such efforts have traditionally been employed in contexts where all students are exploring the same research question (11) or where the content focus has been unilateral (e.g., sole focus on graphing techniques; 21). In this article, we reported on the development and assessment of an interactive CURE statistics workshop that was expressly designed to be flexible in its adaptation to a variety of student-driven research questions and that used both traditional and constructivist methods to enhance students’ understanding of a host of quantitative aspects relevant to engaging in discovery-based science (e.g., statistical analysis, data processing). Data indicated a significant increase in both statistical reasoning abilities and quantitative literacy skills. Furthermore, students self-reported positive affect toward the workshop, and their perceptions of their knowledge of and confidence in applying various quantitative skills likewise increased.

These outcomes are consistent with prior empirical evidence in the literature. In their analysis of student outcomes within a “big data” CURE, Makarevitch and colleagues (19) demonstrated, for instance, that quantitative skills were of utmost importance in constructing representations of complex datasets and for using those data to make informed decisions about future hypotheses to pursue. This recommendation echoes the decades-old argument that teaching statistics within the laboratory itself (rather than as a separate course) improves students’ data analysis and interpretation abilities (e.g., 22). Accordingly, establishing practical connections between students’ own research questions and the methodological lens through which to interpret their findings, as we have done here, appears to address the manifold concerns related to students’ poor perceptions of themselves as quantitative reasoners and scientific researchers (4, 6–8). While this is the case, we propose that future research examine the longitudinal impact of structured statistics interventions, such as the one described in this article, on student success and affect in the discipline. With specific regard to our own research, such studies could, for instance, provide a critical perspective on the relationship between mathematics-based interventions in the biological sciences and the persistence (or lack thereof) of student misconceptions pertaining to quantitative reasoning in the domain. This, in turn, could inform development of future initiatives aimed at addressing observed misconceptions, thereby generating iterative, parallel opportunities to enhance students’ development of statistical reasoning skills.

From a broader pedagogical and personal standpoint, attentiveness to student development of quantitative skills within (and beyond) the context of CUREs is, we contend, an essential facet of preparing individuals for an increasingly competitive STEM workforce (23, 24). Furthermore, enhancing students’ quantitative literacy has been documented as positively impacting their ability to apply mathematical thinking to everyday decision-making tasks (14, 19). This latter claim is anticipated to be of significant educational importance, particularly given the recent emphasis on developing CUREs for nonmajors courses, where scientific literacy is argued to be a critical component of course outcomes (25). While questions pertaining to student outcomes will therefore continue to be of value, we propose also that future research examine the contextual features potentially mediating such observations. If the interactive workshop described herein were to be scaled up, for instance, we might anticipate that variability in GTA instructional behaviors could directly influence students’ acquisition of statistical reasoning skills and mathematical understanding. As we collectively consider these and other questions, the answers obtained will yield promising insights into how to best prepare students to be informed, data-savvy decision makers in both the personal and professional arena.