Figure Facts: Encouraging Undergraduates to Take a Data-Centered Approach to Reading Primary Literature

Figure Facts is a versatile instructional tool designed to help college students tackle complex data figures in the primary literature.

Abstract

The ability to interpret experimental data is essential to understanding and participating in the process of scientific discovery. Reading primary research articles can be a frustrating experience for undergraduate biology students because they have very little experience interpreting data. To enhance their data interpretation skills, students used a template called “Figure Facts” to assist them with primary literature–based reading assignments in an advanced cellular neuroscience course. The Figure Facts template encourages students to adopt a data-centric approach, rather than a text-based approach, to understand research articles. Specifically, Figure Facts requires students to focus on the experimental data presented in each figure and identify specific conclusions that may be drawn from those results. Students who used Figure Facts for one semester increased the amount of time they spent examining figures in a primary research article, and regular exposure to primary literature was associated with improved student performance on a data interpretation skills test. Students reported decreased frustration associated with interpreting data figures, and their opinions of the Figure Facts template were overwhelmingly positive. In this paper, we present Figure Facts for others to adopt and adapt, with reflection on its implementation and effectiveness in improving undergraduate science education.

INTRODUCTION

The ideal undergraduate biology classroom is a hub of active learning in which students engage in the investigative process of scientific research rather than memorizing facts (National Research Council, 2003; Handelsman et al., 2004; Labov et al., 2009; American Association for the Advancement of Science, 2011). Guided explorations of primary literature are especially useful in cultivating an inquiry-based learning environment and enhancing critical thinking skills (Muench, 2000; Levine, 2001; Smith, 2001). Primary research articles not only depict the nature of the scientific process, but also train students to evaluate data critically, improve their scientific writing, and follow the most recent advances in research (Levine, 2001; Kuldell, 2003; Hoskins et al., 2007). The acquisition of these and other information literacy skills increases students’ motivation to pursue research and prepares them to excel in graduate school, medical school, and research laboratories (Kozeracki et al., 2006; Hoskins et al., 2007).

Despite the importance of incorporating primary literature into the undergraduate biology curriculum, reading a scientific paper remains a daunting task for most students. The dense material, filled with unfamiliar terminology, technical details, and complex figures can easily overwhelm a novice reader. Difficult assignments that induce stress and anxiety in students can hamper the learning process (Pekrun et al., 2002). In our personal experiences, we have observed that introductory and upper-level students often approach a research article as they would a textbook, focusing on the narrative of the paper as fact, with the figures subordinate to the text. Undergraduates come to class having underlined and highlighted large portions of the text, yet they are often unable to describe the rationale for an experiment or interpret data presented in the figures. Many students also fail to recognize that research articles often contain elements of persuasion and controversy and should be examined with critical eyes (Gillen, 2006). Thus, it is important that students engage the growing body of primary literature and learn specific strategies to analyze, interpret, and evaluate research papers critically.

A number of methods have been proposed and practiced to assist students with the analysis of primary literature (Janick-Buckner, 1997; Glazer, 2000; Levine, 2001; Gillen et al., 2004; Hoskins et al., 2007; Gehring and Eastman, 2008; Wu, 2009). A common classroom method is the journal club approach, in which one or more students present an article that has been read by the group (Glazer, 2000; Edwards et al., 2001; Kozeracki et al., 2006). A journal club approach, in which students become teachers, has the added benefit of sharpening students’ presentation skills, but the unstructured reading experience allows nonpresenters to engage the material passively and superficially. A second tactic often used by instructors is to design study questions to accompany each paper (Janick-Buckner, 1997; Levine, 2001; Wu, 2009). Study questions ensure that each student identifies the important points of the paper proactively, but this method is often time-consuming for the instructor and requires new questions for each paper, and questions must be crafted very carefully to prevent students from relying on the article text alone.

One particularly effective and novel approach to help students with primary literature is the CREATE method, described by Hoskins et al. (2007). CREATE utilizes a series of four related papers published by the same lab, which allows students to observe how the research process evolves over time. CREATE is unique in that it takes a data-centered approach to reading the papers; large sections of text are withheld from the students, and they answer figure-specific questions that require them to compare and contrast multiple individual panels. Students also predict future experiments and contact authors to gain a personal account of the research process. The in-depth CREATE approach requires a great deal of instructor preparation and relies on a nested set of sequential papers, which may not be practical for all course designs.

The Figure Facts template described here is similar to the CREATE method, in that it intentionally shifts students’ focus from the text to the data by supplying an analytical template with which to interpret each figure. However, Figure Facts is distinct from CREATE, in that Figure Facts takes a more streamlined approach to dissecting the article, often distilling the essential information into one or two pages. Importantly, the Figure Facts approach requires minimal instructor preparation, because there are no paper-specific study questions to write or grade, and the article is given to the students in its entirety, rather than being separated into discrete sections. The instructor-friendly Figure Facts method may be particularly useful for courses that utilize multiple, disparate primary literature papers on a regular basis. Figure Facts is flexible enough that students could easily adopt this method when reading primary literature in any course or scientific discipline. Figure Facts is not intended to replace other highly effective methods for teaching data analysis and interpretation, but may serve as an adaptable, complementary option for the instructor's toolbox.

The primary goal of Figure Facts is to shift students’ focus away from the text of primary research articles and encourage them to spend more time interpreting the data figures. A secondary goal of the template is to provide a structured reading experience that reduces the frustration and anxiety that may serve as a barrier to student learning (Pekrun et al., 2002). To determine whether Figure Facts meets these educational objectives, the lead author (J.E.R.) implemented Figure Facts in an undergraduate cellular neuroscience course for two semesters. In the second semester, the authors measured whether students who used Figure Facts spent more time examining data figures in primary research papers. The authors also tested the assumption that a course based in primary literature improves students’ data interpretation skills. Finally, the authors examined students’ attitudes toward Figure Facts and determined whether this approach alleviates some of the frustration and anxiety that can reduce student learning (Pekrun et al., 2002).

METHODS

The Figure Facts Template

The Figure Facts template is a Microsoft Word table that students fill in electronically as they read a primary research article in preparation for class (Figure 1). The upper section of the Figure Facts template is devoted to the introduction of the paper. Students are asked to read the introduction carefully and fill in four key pieces of information that help them to identify the rationale for the experiments presented in the paper. First, they state the “broad topic” and the “specific topic” addressed by the research. These prompts serve to reinforce the relationship between the research paper and the current course subject matter, while simultaneously narrowing the focus of the paper. Next, the students identify “what is known.” Here, the students summarize previous findings that led to the current study. Listing prior knowledge is especially useful, in that it provides context for the paper and demonstrates continuity of the research process. Students should recognize that research builds on previous findings, and experiments are often based on unanswered questions generated by earlier studies. Finally, students state the “experimental question,” which prompts them to articulate the central question around which the paper's experiments are based.

Figure 1.

The Figure Facts Template. Students were provided with a Microsoft Word file containing a blank table, and they adjusted the number of rows to match the number of figures for each article. Students filled in the table using abbreviations and short phrases to save space and minimize distraction from the research article. Students uploaded the completed file to an online course-management system and brought a paper copy to class, which allowed them to recall information readily and to take notes during class discussion.

The main body of the template is devoted to the data figures of the primary research article. For each figure panel, students describe the specific experimental technique performed by the investigators. In the parallel column, students state the result and conclusion that can be drawn from each experiment. When filling out the template, students are instructed to dissect each figure before reading the corresponding text in the results section. They are asked to use their own words to describe the methods and results, rather than paraphrasing the figure legend or copying information from the text. Having students use their own words encourages students to interpret the data independent of the author's interpretation. Students are also advised to use short phrases and abbreviations, rather than complete sentences, to minimize distraction from reading and understanding the paper itself.

Implementation in the Classroom

The lead author (J.E.R.) developed and used Figure Facts in two iterations of a Davidson College Cellular and Molecular Neuroscience course (BIO 333), which included 14 biology and neuroscience majors in Spring 2011 and 16 students of a similar demographic in Spring 2012. The elective course consists of two 75-min gatherings per week for 16 wk. Each class period is a fluid mixture of lecture, discussion, and data analysis. All reading assignments come from primary literature, scholarly reviews, and online resources. The primary research articles usually contain microscopic images and/or complex graphical representations of averaged data sets. When reading these articles in preparation for class, students are required to complete a Figure Facts template (Figure 1). Students type their responses to the prompts in the electronic document using word-processing software.

Prior to the class meeting, students upload the completed template into an assignments folder in Moodle, a course-management system. Uploading Figure Facts prior to class requires students to make a significant effort to understand each figure before the group meeting, rather than skimming the paper briefly before class. The upload requirement also follows the principles of just-in-time teaching (Novak et al., 1999), in that it allows the instructor to assess each student's level of understanding prior to class and adjust group discussion to suit the students’ needs.

During class, each figure is projected using Microsoft PowerPoint, and we examine the data closely as a group. With PowerPoint, the instructor can divide more complex figures into multiple parts, which prevents students from becoming overwhelmed by too much information at once. Projecting the figures guarantees that everyone is looking at the correct figure panel and allows students to interact with the data using laser pointers that are passed around the room.

The instructor asks questions to structure the conversation, but the students are expected to articulate the methods and results of each figure, identify weaknesses in experimental design, and propose future experiments. Students are instructed to bring a hard copy of their completed template to class, so they can refer to their notes as we examine the data. Students are encouraged to write on the template and fill in any missing information as we go along. Later, the template serves as a condensed study guide to aid the student in preparing for quizzes and exams.

Collectively, Figure Facts are worth 10% of each student's course grade, which provides sufficient incentive for students to complete each assignment. To minimize the time required to grade the templates, we scored them on a scale of 1–5. While the students are not expected to interpret every figure correctly, they are asked to provide evidence that they made a significant effort to understand each experiment. No part of the template can be left blank, and each box has to contain at least a best guess or a question related to the figure panel. With a cursory examination of each template, the instructor is able to determine whether the information is complete, contains sufficient detail, and is written in the student's own words. On average, each template should take only 1 or 2 min to examine and grade. The first template of the semester is scored but not included in the grade calculation, which allows students to become familiar with expectations and receive feedback from the instructor.

Assessment Design

The primary goal of Figure Facts is to shift students’ focus away from the text of research articles and encourage them to spend more time examining the figures, thus sharpening students’ data interpretation skills. A secondary goal of Figure Facts is to provide a structured reading experience that will reduce some of the frustration and anxiety students often experience when learning to read primary literature. To determine whether Figure Facts accomplishes these educational objectives, we designed a three-part assessment plan. First, to determine whether Figure Facts increases students’ attention to data figures, we performed a time-on-task assessment, in which students documented their paper-reading activities before and after intervention with Figure Facts. Specifically, in weeks 1 and 15 of the Spring 2012 semester, students recorded the amount of time they spent reading each section of a primary research article. They were asked to differentiate between the time they spent examining data figures and the time they spent reading about those figures in the text. We also recorded the number of handwritten notations that students made in the margins of each figure panel to ascertain the levels of student engagement with the visual data. It is important to note that students were not required to fill out a Figure Facts template for the week 1 or the week 15 assessment. This element of the assessment design allowed us to determine whether students spent a greater percentage of time examining figures after using Figure Facts for one semester, even in the absence of the template. Student data from weeks 1 and 15 were analyzed statistically using a paired t test.

Second, to determine whether a course grounded in primary literature sharpens students’ data interpretation skills, we administered three ungraded skills tests at regular intervals during the Spring 2012 semester. (The skills tests are available upon request and are appropriate for biology majors with prior basic knowledge of neuron structure and function and previous experience interpreting graphs.) Each 30-min test consisted of two figures containing microscopic images of neurons and graphs of averaged data sets. Students were asked to examine the figures and then identify the true statements from a list of possible conclusions. All students had completed the prerequisite course, BIO 111 (Molecules, Genes, and Cells), in which they discussed neuron structure and function, examined microscopic images, and interpreted graphs. Therefore, each student had sufficient content knowledge to complete the assessment. Each test contained 14 answer choices and 5 correct answers, yielding a score range of −14 to +5. The tests were administered during class time to maximize students’ attention and effort. Student performance was tracked using anonymous identification numbers, and data were analyzed using paired t tests.

We also administered the skills tests to a control group of 15 Davidson students who had completed the prerequisite course, BIO 111, and thus had experience with neurons, microscopic images, and graphs. Students who completed the tests in identical order in one sitting showed no significant improvement between the first and second test (t(13) = 0.18, p = 0.86) or the first and third test (t(13) = 1.94, p = 0.08). These control data suggest that the three skills tests are comparable with regard to difficulty, and any improvement exhibited by BIO 333 students over the course of the semester is not attributed to a more difficult first test.

Third, to assess students’ opinions of the Figure Facts approach, we administered anonymous midsemester surveys in both iterations of the course. The midsemester survey utilized a 5-point Likert scale, which asked students to agree or disagree with statements pertaining to the use of Figure Facts. In Spring 2012, we added pre- and postcourse surveys to assess students’ experiences and attitudes toward reading primary literature in general. Specifically, we used a 5-point Likert scale to ask students whether they considered reading primary research articles to be a frustrating and/or stressful experience, and we examined whether a semester of guided literature exploration reduced negative emotions associated with this complex task. The overall IRB study participation rate for the three-part assessment was 100%, with some student scores and opinions omitted due to absence from class or incorrect completion of the assignment. Students consented to be included in the study by completing and submitting the precourse survey described above.

RESULTS AND CONCLUSIONS

Time-on-Task Assessment

The primary goal of Figure Facts is to shift students’ focus away from the text of primary research articles and to encourage students to spend more time interpreting the data figures. To determine whether Figure Facts was associated with increased student attention to data interpretation, we performed a time-on-task assessment in which students documented their paper-reading activities before and after exposure to Figure Facts. In week 1, before students were introduced to the Figure Facts concept, students spent approximately 80% of their time reading the text and 20% of their time examining the data figures (Figure 2A). In week 15, after completing eight Figure Facts templates, students spent only 60% of their time reading the text, and doubled their time examining data to 40%. This substantial increase in the percent time spent interpreting data figures was statistically significant (t(13) = 5.5, p < 0.001).

Figure 2.

Students spent more time examining data figures after using Figure Facts for 15 weeks. (A) In weeks 1 and 15, students recorded the amount of time they spent reading the text of a primary research article vs. examining the data figures. (B) The number of notations made in the margins of each figure panel was also recorded at weeks 1 and 15. Error bars, SEM; ***, p < 0.001 by paired t test; week 1: n = 14; week 15: n = 16. Two students were omitted from the week 1 analysis due to incorrect completion of the assessment.

In addition to tracking data interpretation as a percentage of time, we recorded the number of handwritten notations that students made in the margins of each figure panel (Figure 2B). No instructions or grades were given for handwritten notes on the paper, so the students had no external incentive for making these notations. In week 1, students made notations on 10% of the printed figure panels, whereas in week 15, students marked 38% of the figure panels (t(15) = 4.4, p < 0.001). Therefore, students appeared more actively engaged in examining figures for comprehension in week 15, as indicated by the nearly fourfold increase in handwritten notations on hard copies of the research papers. It is important to note that students were not required to complete a Figure Facts template for the week 15 assessment. Therefore, prior experience with the template was sufficient to induce a significant and measurable shift in students’ paper-reading habits.

Data Interpretation Assessment

An underlying assumption of the Figure Facts approach is that students who practice reading primary research articles for comprehension will sharpen their data interpretation skills. Specifically, students who examine data figures repeatedly will improve their ability to comprehend complex graphical representations of data and draw accurate conclusions. To test this assumption we administered three skills tests at regular intervals throughout the semester. Each test consisted of two figures containing microscopic images of neurons and graphs of averaged data sets. Students were asked to examine the figures and identify true statements from a list of possible conclusions. In week 1, the average student score was −1 (on a scale of −14 to +5), indicating that most students tended to choose false conclusions more often than they chose correct conclusions (Figure 3; n = 16). In week 9, after students had analyzed eight primary research papers in detail, the average score improved significantly (t(14) = 4.22, p < 0.001) to +1. By week 15, students’ performance trended upward to +2, but the difference between week 9 and week 15 was not statistically significant (t(1) = 0.94, p = 0.36). The measurable improvement observed from week 1 to week 9 suggests that regular, structured primary data analysis improves students’ abilities to interpret novel data sets.

Student Attitude Surveys

To assess students’ attitudes toward Figure Facts and primary literature in general, we asked students in both iterations of the course to complete anonymous midsemester surveys. Eighty percent of the students agreed or strongly agreed that Figure Facts helped them structure their reading of primary research papers (Figure 4A). One student remarked that dividing the readings into discrete, manageable steps reduced some of the anxiety associated with tackling unfamiliar research papers. Ninety percent of students agreed or strongly agreed that Figure Facts helped them focus on the data, rather than the text of the paper. One student wrote the following comment on the anonymous midsemester survey:

I really appreciate the Figure Facts … I used to skip over the figures in articles because they were too difficult to understand, but forcing me to focus on the data is teaching me a better way to read and understand research.

Figure 4.

Students had favorable opinions of Figure Facts. Anonymous (A) midsemester survey and (B) pre- and postcourse survey results from students across 2 yr regarding the use of Figure Facts. Students scored their agreement with each statement on a Likert scale of 1 to 5 (1 = strongly disagree; 2 = disagree; 3 = neutral; 4 = agree; 5 = strongly agree). Error bars, SEM; *, p = 0.035; ***, p < 0.001; n = 29. One student survey was omitted due to absence from class on the day of survey administration.

This comment indicates that Figure Facts generates the desired educational goal, which is to guide students away from their dependence on the text and encourage them to persist in their efforts to interpret the data.

The midsemester surveys also addressed issues of class preparation and discussion. Eighty percent of students agreed or strongly agreed that Figure Facts assignments encouraged them to read the paper more closely in preparation for class. Requiring students to upload their template and assigning a grade to their effort encouraged students to regard the paper analysis as a high-priority exercise, rather than a passive reading assignment that they could skim right before class. Sixty percent also agreed or strongly agreed that the template helped them organize their thoughts during class discussion. Long moments of silence during discussion times were rare, because students could easily refer to their prepared comments if they had trouble recalling information pertaining to the figure at hand. Students were able to recall basic information quickly, which gave us more time to discuss the benefits and drawbacks of each experiment, critique the experimental design, and propose improvements to the study. Finally, 80% of students agreed or strongly agreed that the template made our classroom discussions more productive, suggesting that a well-prepared and well-organized approach to discussing primary literature in the classroom is favored by instructors and students alike.

In Spring 2012, we administered pre- and postcourse surveys to assess the levels of stress and anxiety students experience when reading primary literature. In week 1, 47% of students agreed or strongly agreed that reading primary research papers was often a stressful experience for them, and 58% reported that they became frustrated when trying to understand papers (Figure 4B). By the end of course, only 12% of students agreed that reading research papers was a stressful experience and only 19% reported feeling frustrated (Figure 4B). These findings indicate that repeated exposure to primary data, combined with a structured reading approach, is associated with reduced student anxiety and frustration. Given that heightened stress and anxiety correlates inversely to learning gains (Pekrun et al., 2002), reducing frustration may aid students significantly in learning the data interpretation skills that are central to evaluating research papers. This finding complements a recent report by Hoskins et al. (2011), which shows that guided literature explorations increase students’ overall confidence in their ability to interpret data.

At the end of the semester, students also completed standard, college-issued course evaluations, which asked students to comment on specific elements of the course that they found helpful to the accomplishment of the course's goals, as well as areas for improvement. Half of the students spontaneously provided positive comments pertaining to Figure Facts in this section of the form (Table 1), and no negative comments pertaining to Figure Facts were encountered. These unsolicited comments suggest that many students maintained a favorable opinion toward Figure Facts at the end of the semester and regarded the innovation as a highlight of the course.

Table 1.

Selected student responses to the following end-of-semester course evaluation question: What elements of the course and instruction did you find most helpful to the accomplishment of the course's goals?

“Figure Facts was a great teaching tool for article comprehension.”
“Figure Facts were helpful in learning how to read modern science writing.”
“Figure Facts were helpful in forcing us to finally analyze papers in detail.”
“Figure Facts kept me up-to-date and organized.”
“Figure Facts were helpful in understanding the primary sources.”
“Figure Facts actually really helped me improve my scientific reading abilities.”
“Figure Facts were challenging, but a really helpful and unique way to learn about experiments, and a great way to study.”

DISCUSSION

Future Improvements

The preliminary use of Figure Facts in the classroom was successful in that students shifted their focus from the text of the articles to the data figures (Figure 2), improved their ability to interpret complex data sets (Figure 3), and experienced reduced frustration and anxiety with regard to reading primary literature (Figure 4). However, it is interesting to note that, on the midsemester survey, only 38% of students agreed that Figure Facts helped them to identify how the study related to previous research (Figure 4A). From this finding, we conclude that the “what is known” prompt in the table is not effective in helping students to understand how the paper relates to previous findings. Perhaps simply changing the name of this prompt from “what is known” to “previous research” may make the link between the current paper and earlier experiments more obvious. Student feedback also indicated that we should spend more time discussing the previous findings at the beginning of class in order to place the current experiments into context. It may be especially useful to assign two or more related papers, as advocated by the CREATE approach (Hoskins et al. 2007), to better illustrate the continuity of the research process.

Figure 3.

Students interpreted novel data sets more accurately at week 9. At weeks 1, 9, and 15, students were asked to examine a collection of microscopic images and graphs and identify true statements from a list of possible conclusions. Positive values indicate more correct than incorrect answers. Box represents the middle 50th percentile, line represents the median, and whiskers represent minim-um and maximum values. ***, p < 0.001 by paired t test; n = 16.

A second potential improvement to the Figure Facts template would be to move the “experimental question” prompt to the bottom of the template. After spending a great deal of time on the details of each figure, it may be useful for students to step back, state the main conclusion of each figure, and confirm the experimental question being addressed by the authors. By stating the main idea at the end of the discussion, students may be able to articulate the take-home message of the paper more clearly and gain a greater appreciation for the complex series of experiments required to generate evidence in support of a single hypothesis. It may also be beneficial to ask, “What is the next experiment?” at the bottom of the Figure Facts template to encourage students to think about appropriate future experiments in preparation for group discussion.

An important caveat of this study is that it was performed with small, highly selected samples of students who had close interaction with a single instructor. Student and instructor motivation may influence positive assessment gains and student attitudes to some degree, so the effectiveness of Figure Facts should be assessed on a larger scale in different classroom and campus settings before generalizations can be made. The results of the data interpretation skills tests should also be regarded as preliminary, due to the small scale of the assessment. Science educators could benefit greatly from a larger-scale study examining the impact of primary literature exposure on students’ ability to interpret complex data sets. Finally, while the overwhelmingly positive student feedback highlighted in Figure 4 and Table 1 is encouraging, it would be interesting to determine whether students are inclined to use Figure Facts on a voluntary basis, both in the classroom and in an independent research setting.

Summary

The Figure Facts template provides a structured reading exploration for undergraduate students as they learn how to analyze primary literature. We found that Figure Facts does indeed increase students’ efforts to interpret data figures, while simultaneously reducing the frustration often associated with this task. With this increased attention to data figures, students improved their data interpretation skills when presented with novel data sets. With basic figure content mastered in advance, students could devote more class discussion time to critical analysis of experimental design, alternative interpretations of results, and potential future experiments. Figure Facts can be adapted easily to any research article in any science course, and is one of many tools that instructors may use to demonstrate the investigative nature of biology through primary literature.