Learning Experimental Design through Targeted Student-Centric Journal Club with Screencasting

Abstract

Knowledge and application of experimental design principles are essential components of scientific methodology, and experience with these skills is fundamental for participating in scientific research. However, undergraduates often enter the research laboratory with little training in designing and interpreting their own experiments. In the context of a research university laboratory, we designed a journal club training exercise to address this need. Students were instructed on methods for interpreting scientific literature using a screencast, a digital recording of a slide presentation narrated by an instructor. Students subsequently examined a series of research publications with a focus on the experimental designs and data interpretation in a two-session group discussion journal club format. We have found this approach to be an efficient and productive method for engaging students in learning about principles of experimental design and further preparing them for success in laboratory research.

Undergraduate research opportunities have numerous positive educational outcomes and encourage students in STEM career paths (Lopatto, 2004, 2007; Russell et al., 2007). To fully engage in scientific research, students need to be skilled in experimental design and data interpretation. Developing these skills is a major goal of undergraduate science education, and enhancing the rigor and effectiveness of science learning are a national priority in higher education (Woodin et al., 2010). Laboratory research draws from the published scientific literature to establish hypotheses and conduct experiments. However, undergraduates often start in the research laboratory with little to no experience with reading primary scientific literature; introductory undergraduate course instruction often focuses on scientific knowledge and the outcomes of scientific experiments. Students may have basic exposure to reading scientific journal articles in classes, but often have had little to no formal training on how to interpret scientific data or utilize published findings in designing an experiment. While the majority of science faculty strongly agree that students should be taught scientific process skills such as interpreting data and designing experiments, most do not spend enough time in classes to do so because of priority given to cover content material (Coil et al., 2010).

Based on these findings and our experiences training dozens of undergraduates in independent research projects, there is an ongoing need for improved analytical training of undergraduates (e.g., analysis of experimental design, data interpretation), and undergraduate research opportunities that include such training can address this need. While giving students scientific journal articles is one of the most direct methods for initially connecting them with their project, this standard practice may not be a productive approach because students usually lack training and experience to understand this literature. Learning to read scientific journal articles is a foundational skill for participating in research; teaching these skills can enhance student productivity in the laboratory. Additionally, learning scientific reasoning through analyzing scientific journal articles is an effective tool in a number of classroom settings, including discussion-based journal clubs (Glazer, 2000; DebBurman, 2002). Journal clubs may be effective for training undergraduates in such analytical thinking due to the collaborative learning and social peer support in small group discussions (Johnson and Johnson, 1996).

One challenge in teaching students experimental design is having the time necessary to do so in the face of requirements for covering content in the classroom or completing experiments in the laboratory. Such teaching would ideally maximize the efficiency of instructor-student interactions while enabling the student to have extensive interaction with the material. One way to address this issue is employing a “flipped classroom” approach (Bishop and Verleger, 2013), administering content electronically and focusing in-person time on discussion and interactive training. In terms of pedagogy, this kind of blended learning approach (online content combined with face-to-face interaction) promotes critical thinking and meaningful educational experiences (Meyer, 2003; Garrison and Kanuka, 2004). Flipped classroom approaches enable active learning with greater student-instructor interaction in the classroom. Such approaches have been consistently reported to increase student learning and achievement in undergraduate science courses compared to traditional in-class lecture formats (Moravec et al., 2010; Deslauriers et al., 2011; Roehl et al., 2013).

To better connect students with their research projects while developing their scientific reasoning skills, we established a journal club using screencasting with the explicit goal of teaching students principles of experimental design. A screencast is a digital recording of a slide presentation that students can view outside of lab on their own time and pace. Here we describe an example format derived from five iterations of our undergraduate journal club. Our intent is to provide examples that enable other researchers to utilize blended learning approaches in support of undergraduate student development in experimental design and data interpretation.

Our overall goal was to increase undergraduate student proficiency in designing experiments (e.g., set specific goals with proper experimental controls) and their analysis and interpretation of data. Our learning objectives were to (1) enable undergraduates to plan and discuss experimental design in lab (2) enable undergraduates to find and understand primary literature articles, (3) maximize efficiency of student learning (e.g., limit time required to complete training), and (4) enhance work relationships between lab members outside of time in lab. Given the full-time research commitment of the laboratory staff and the complexities of undergraduate class schedules, we also sought to make instructional time and preparation as efficient as possible.

MATERIALS AND METHODS

To pursue these goals, we engaged undergraduate students with a sequential set of materials that spanned from familiar and simple (i.e., research data from our laboratory) to less familiar and more complex (e.g., scientific journal articles with and without schematic cartoon diagrams) to facilitate progressive growth in experimental design. We created a two-session journal club to connect undergraduate students to experimental design principles through reading primary literature (Figure 1). To maximize scheduling flexibility for students to complete these extracurricular activities and streamline preparation time, we employed a “flipped classroom” approach utilizing a screencast to deliver content. We developed a 20-minute screencast for students to view before our first in-person meeting to demonstrate experimental design concepts and interpretation of data visualization (See Supplemental Material 1). Creating a screencast involves (1) making a slide presentation, (2) preparing an associated lecture, and (3) making a video recording on a computer of the slides + voiceover using screen capture software. In addition, before the first journal club, we distributed a handout on critical reading of scientific papers to highlight elements for students to look for in their reading (See Supplemental Material 2). We then gave students a handout on how to use the Google Scholar search engine (See Supplemental Material 3) and tasked students with submitting links of five papers of interest to the instructor; this component enabled students to practice finding papers on their own and provided material for a student-selected paper to discuss in our second journal club session. Instructions for finding papers were open-ended (i.e., students choose based on interest), focused more on getting students to look into the literature and less on pre-evaluating the experimental design of studies.

Figure 1.

Flowchart of targeted journal club activities.

The two journal club sessions were each approximately two hours in length. The instructor-led first journal club involved reviewing the sample figures from the screencast and then discussing the first scientific paper (selected by the instructor to highlight aspects of experimental design). The student-led second session focused on discussion of the scientific journal article selected from papers submitted by students (distributed following the first journal club session). Although time-intensive, we have found that this extended meeting format gives students the space and time necessary to wrestle with interpreting the data and to vocalize their understanding of primary scientific literature. Based on personal observations, these sessions seemed most effective when there were at least two instructors and two students per session. Having multiple instructors enabled more comprehensive discussions in that each instructor could ensure all topics were addressed and redirect conversations to include important details. Having more than one student created a peer environment that encouraged students to participate in putting forth their opinions even when students were unsure of their own understanding. Conversations from these group sessions were often continued in subsequent laboratory interactions, reinforcing and expanding concepts and student understanding.

While the theory of experimental design is the focus of this journal club structure, technology can be useful for increasing learning efficiency and student involvement (Garrison and Kanuka, 2004). Originally, we held an additional session to introduce principles of experimental design in a lecture format. We progressed to using a screencast to convey content to students in a way not constrained by their physical time in lab, a useful addition to focus their lab time running experiments. Specifically, we used the recording function in Microsoft PowerPoint combined with free sound recording software (Audacity) to generate individual sound clips per slide. This approach can be advantageous compared to simply recording a presentation in PowerPoint because sound clips do not need to be collected consecutively and can be revised into an efficient presentation. One issue with this approach is that sharing the files online can be challenging due to their large size. However, screencast software options are rapidly evolving to more user-friendly platforms (e.g., Jing, Screencast-o-Matic, Camtasia, ScreenFlow, QuickTime Player, Google Hangouts ) that allow for free screencast recordings that can be easily dispersed online (e.g., upload to YouTube, store in Google Drive). For literature searching, we had students work with Google Scholar and utilize the library links feature to be able to find articles accessible to the university. We recommend instructors utilize search engine/databases that (1) are simple to use for novice researchers and (2) they want students to use in their research/class. We use Google Scholar because of the user-friendly interface, the inclusion of citation statistics (i.e., can focus on highly cited works), and broader content searching capacity (e.g., those sources not filed in PubMed; conference proceedings, very recent publications). While databases like PubMed contain greater depth of scholarly sources (Bramer et al., 2013), Google Scholar is sufficient for introductory literature searches (Falagas et al., 2008; Nourbakhsh et al., 2012). Informal student feedback indicates they find Google Scholar easy to understand and use. For scheduling the group discussion sessions, we utilize online scheduling tools ( Doodle ). An important aspect not to be overlooked is understanding student aptitude with the technology. Although students are often skilled in many personal uses of technology, they initially may not be proficient or comfortable using these specific tools. Instructors wanting to employ these technologies may need to explicitly instruct students on their use.

PEDAGOGY

Here we describe each of the elements of the targeted journal club format; each can easily be adapted to other laboratory settings based on the specific learning goals and scientific expertise of instructors. The first two elements are discussed in the screencast.

1) Introduction to experimental design; creating and interpreting a figure based on laboratory data

We begin with a description of how to test a hypothesis through a series of experiments followed by the construction of a figure based on data from research projects in the laboratory. This approach is useful in highlighting the transition from experimental question to hypothesis and finally to a defined strategy for testing a hypothesis. In our screencast, each student goes through the following elements:

1. Define experimental controls (i.e., what are vehicle/injection, positive, and negative controls?)

2. Identify experimental variables in given experiment (i.e., use scenario from lab/student project; which factors in the experiment could affect the outcome and be manipulated?)

3. Design set of experiments to isolate variable of interest with appropriate control experiments

4. Build figure from data (i.e., bar graph of data from multiple experiments). Explain how each part of the figure conveys a piece of information and then practice interpreting hypothetical results when individual components of the figure are changed (e.g., from screencast, first example = specific factor/bar shows experimental change, second example = same factor/bar shows no change).

Our screencast focused on using control experiments to isolate a specific variable and exclude alternative explanations, addressing the question “how can we know that observation ‘A’ is due to factor ‘B’ and not factor ‘C’?” (Figure 2A). In our specific laboratory context, we were examining how a particular protein (a steroid receptor subtype) may or may not bind to DNA sequences around genes that regulate the mRNA transcription of this gene. We discussed appropriate control experiments for three experimental variables/questions (Treatment = does steroid treatment regulate a gene’s mRNA level?; Receptor = which of two steroid receptors mediates the observed mRNA change?; Direct/Indirect = is the mRNA change a direct effect of the steroid receptor on the gene or an indirect effect through regulating the mRNA of another protein?) and how corresponding results would be added to a figure (Figure 2B, 2C). This sequential addition of experimental data in the construction of a figure also emphasizes how each component of a figure conveys an experimental result. Overall, this exercise demonstrates connections between experimental planning, results, and data visualization in a single example that has content familiar to the students.

Figure 2.

Diagrams of experimental design theory, sample experimental design flowchart, and corresponding projected data.

(A) Relationships of outcome and experimental variables to consider in designing experiments

(B) Diagram of possible relationships options for three factors testing gene expression regulation; corresponding experiments and appropriate controls are based on this logic

(C) Screencast slides conveying data of gene being directly regulated in cell culture by a specific steroid receptor based on four experimental results; in this example, gene “A” is regulated by steroid treatment (yellow bar) compared to vehicle treatment (light blue bar), regulated by the receptor-selective agonist (orange bar), not regulated by co-treatment of steroid with an antagonist (teal bar), and regulated by steroid treatment in the presence of protein synthesis inhibitor (green bar).

2) Applying knowledge of experimental design to understand new information; interpreting data from a scientific journal article

Having examined the process by which experiments are conducted and how research data can be compiled into figures, the screencast next presents a single figure from a scientific journal article with a research theme related to an undergraduate student project in laboratory. The students first view the entire figure and then go through each panel of the figure independently, analyzing the data in a manner similar to our laboratory project analysis (emphasizing control experiments, using similar vocabulary and language to reinforce connections). The screencast then describes how each part of figure supports the figure title, emphasizing control experiments and which experimental variables each control addresses, and then displays the entire figure again while summarizing the findings. Inclusion of topics from a student project may also instigate additional connections and conversations relevant to their own research. For additional practice, a single figure from another paper is included for students to analyze independently and then discuss at the first journal club session.

3) Applying knowledge of experimental design to understand a research study; discussing a scientific journal article containing visualization of data and experimental design

After students have viewed the screencast, we meet for our first group journal club session to go through the practice questions and discuss the first paper. To enable students to interpret an entire paper on their own, we carefully select a short paper (3–4 figures) that (1) focused on a topic personally relevant to students, (2) defined the research methods to an appropriate level for undergraduate students, (3) clearly communicated background information and results, and (4) contained explicit visualizations of techniques and experimental design. For this purpose, we have used a scientific journal article that describes how caffeine modifies a key set of neurons in the brain (Simons et al., 2011). The inclusion of experimental design in figures is particularly important for connecting students with topics and techniques unfamiliar to them. This paper does not have to be directly linked to experiments in the laboratory. In our experience, reading about unfamiliar techniques helps students develop the generalized ability to interpret experimental data. Students read the paper on their own time; instructors lead a guided discussion on the paper. Discussion goals include (1) understanding the scientific question and motivation for the research, (2) having students interpret all figures and (3) highlighting experimental design elements (i.e., control experiments, transitioning scientific questions into lab experiments, how the experimental design connects each figure to next figure).

4) Developing independent capacity to interpret scientific findings; group discussion of student-selected paper

Our second group journal club session is a student-led discussion of a paper of their choosing. To extend student engagement with primary literature, we select a paper from their submitted lists that (1) is appropriate for their current level of technical and biological understanding, and (2) will test their knowledge of experimental design while affording them opportunities to practice their analytic skills. In contrast to the first session, students present the material and go through each figure of the paper in a manner similar to the first session, while now instructors are primarily question generators. This approach attempts to empower the student to take responsibility for understanding the paper by leading and teaching other lab members.

DISCUSSION

We have employed this journal club format in our lab on multiple schedules, including late semester, over the summer, and early semester. In our experience, the early semester version was most successful because that schedule (1) trained students in helpful skills as they entered the research lab and (2) enabled early interaction between students and instructors that supported future conversations and actual experiment planning across the semester. For journal clubs during the academic year, we have held sessions in the evening with a meal. Having journal club during the dinner hour is usually convenient for scheduling, and the format of a meal creates an environment that is inherently relaxing and encourages student participation. From our experience, this journal club format is particularly useful for full-time researchers mentoring undergraduates (e.g., postdoctoral fellows, graduate students, research technicians) due to the concise schedule.

One challenge for students in using web-based content is the issue of electronic multi-tasking, which can hinder learning (Rosen et al., 2013; Sana et al., 2013). We do not know the specific details of how students watched the screencast and whether they were doing other tasks simultaneously (e.g., social media). We attempted to help students interact with the content by including example figures that were then discussed in the in-person session. Their contributions indicated that they had watched the screencast and retained pertinent information (i.e., could answer the questions in discussion).

As described in the methods section, instructions for finding papers were open-ended (i.e., students choose based on interest), focused more on getting students to look into the literature and less on pre-evaluating the experimental design of studies. In retrospect, that approach was not the most productive strategy. Students sometimes identify articles based on an interesting topic but lacking in terms of discussing experimental design (e.g., experimental design not well described, figures overly simplistic, not enough figures for discussion). Focusing the instructions on finding articles that exemplify experimental design features may also engage students further in initial analysis of the article.

This methodology can be beneficial to students both specifically for a given context (e.g., lab makes screencast on their projects) and generally by using the Supplemental Material screencast. For student researchers in a laboratory, content based on their project is relevant for both skill development and topic information. However, the screencast discussed in this article (e.g., about glucocorticoids and astrocyte RNA regulation) can be useful for any student classroom if content level appropriate. Students will still able to follow along and learn how to interpret data and experimental design in classes focused on other topics besides stress biology (e.g., this screencast successfully used in a developmental neurobiology course). Our screencast file is not modifiable, but the general approach is definitely customizable. The basic construction of a screencast involves using software to record voice/slides of lecture on a computer and posting the file online accessible to students (no technical support used/required; store file on web e.g., Google Drive, Dropbox, learning management software/Blackboard). Editing of the screencasts can be done with certain software and raises the level of difficulty for implementation (e.g., ScreenFlow). The technical requirements thus depend on how formal a recording is needed (e.g., short informal discussion vs. edited formal presentation).

This journal club format can be expanded to include additional components as well. For example, when we went through the journal club sessions during the summer, full-time undergraduate researchers had more time available and each presented a paper on their own in more traditional journal club presentations. This format was also used in an upper-level neuroscience lecture course with slight alterations (12–14 students). The class version utilized the screencast with only the first session in-class discussion. Additional scientific papers were examined in the course on a weekly basis, enabling repeated practice in analyzing scientific data and experiments. In the classroom setting, students were divided into small groups to enable every student to discuss the material with peers (3–4 students/group); they were instructed to discuss the screencast and come to a consensus on their answers to the practice figures. The entire class then talked through their findings and reasoning in a manner similar to the research laboratory discussions.

Although we have not quantitatively assessed the impact of this methodology, our experience suggests this approach has a substantial impact upon student engagement and scientific development. The journal club discussions did have structure to focus on student understanding. For example, students were asked with parallel language to the screencast about experimental design of papers (e.g., identify controls, summarize the meaning of each figure) after going through the practice data/figures from the screencast. Informally, multiple self-reported feedback exercises indicate that the experience was helpful for students: (1) in end-of-semester reflection essays for undergraduates researching in lab, all students who took part in this method positively mentioned the journal club training in some way, and (2) in mid-semester feedback questionnaire for the upper level class, the answers of all students to the question “what are 3 important things you have learned thus far?” included some reference to improvement in how to interpret experiments and data in scientific literature.

Understanding experimental design and data interpretation are critical skills for active participation in modern neuroscience (Akil et al., 2016). We have described a student-centered journal club to teach principles of experimental design through analyzing scholarly scientific articles. This simple yet powerful format provides undergraduates first-hand experience with experimental design and finding relevant journal articles. In our experience, this approach enhances student knowledge of the scientific process while establishing vocabulary and group discussion relevant to student projects. This exercise helps instructors evaluate student ability and establish a set of skills important for participation in scientific research. In the laboratory or classroom setting, we hope that these kinds of activities can be used in undergraduate scientific training because of the integral nature of understanding and applying principles of experimental design in scientific careers.