Criticism and Interpretation: Teaching the Persuasive Aspects of Research Articles

Abstract

Research articles are an excellent tool for promoting active learning about the scientific process. One difficulty in teaching research articles is that they address a professional audience and often seek to be persuasive as well as informative. This essay discusses pedagogical strategies that are intended to help students differentiate the purely informative aspects of research articles, such as descriptions of the methods and results, from the persuasive aspects, such as interpretation of results and critical evaluation of the work of other scientists.

THE IMPORTANCE OF PERSUASION IN RESEARCH ARTICLES

Biology teachers increasingly use research articles in their undergraduate courses (Woodhull-McNeal, 1989; Pall, 2000; Levine, 2001; Smith, 2001; DebBurman, 2002; Mulnix, 2003; Gillen et al., 2004; Russell et al., 2004). This trend arises from current pedagogical emphases on scientific process and active learning (National Research Council, 2000, 2003), and it is fueled by the recent widespread online availability of primary literature (Campbell, 2004). Research articles are powerful tools for promoting active learning, and in particular, they can encourage authentic scientific thinking.

Although research articles have tremendous pedagogical potential, there are significant challenges to using them in undergraduate classrooms. In considering these challenges, it is helpful to differentiate between the informative aspects of research articles, such as the description of methods and results, and their persuasive aspects, such as the interpretation of results and the critical analysis of the work of others. Regarding the informative aspect, subject material and terminology can present difficulties in the classroom (Muench, 2000). Instructors are generally aware of these problems and can help students by providing appropriate secondary sources and by explaining unfamiliar concepts. In contrast, the persuasive aspects of research articles may be more difficult to address in the classroom. Nevertheless, there is substantial pedagogical value in exploring this aspect of articles.

My interest in teaching about the persuasive aspects of research articles arises from experiences using primary literature in undergraduate classes ranging from nonmajors courses to upper-level seminars for majors. I have been surprised to find that most students, even nonscience majors, can understand the informative aspects of research articles when I provide appropriate support and encouragement (Gillen et al., 2004). However, even when students have mastered the subject material in an article, they often still face difficulties in developing authoritative critiques. Some students question whether critical evaluation is necessary or even possible. Others advance only their personal opinions or engage an article only peripherally. Finally, many students simply lack the strategies needed to construct authoritative critiques. I believe that these difficulties arise partly because students struggle with the persuasive nature of research articles.

A comparison with textbooks underscores the difficulties that students face when first reading research articles. Textbooks are written mainly to deliver agreed-upon, objective facts to a student audience. The main task for readers is to understand straightforward factual statements. In contrast, reading a research article is more complex. The audience of an article is not students but rather professional scientists, and its purpose is not only to communicate facts but also to address controversies and make arguments (Bazerman, 1988). To effectively read research articles, students must not simply elevate their existing reading skills; they need completely new strategies and approaches. Although developing the new skills is a hurdle for students, this process is also a key to engaging students in the scientific process.

A FRAMEWORK FOR READING RESEARCH ARTICLES

Over the past several years, I have been developing strategies for introducing the persuasive aspects of research articles to my students. My thinking on this topic has been strongly influenced by interactions with those who teach outside the sciences. In this essay, I apply a framework for reading described by a teacher of English literature to the teaching of biology research articles. In Textual Power, Robert Scholes distinguishes between three different stages in reading literature: reading, interpretation, and criticism (Scholes, 1986). There are undoubtedly differences between reading literature and scientific research articles, and Scholes' framework cannot be applied directly to the latter. However, I have found that the concepts of reading, interpretation, and criticism are useful tools. In particular, I believe that students who can identify interpretation and criticism, in the writing of others and in their own thinking, are well positioned to understand the persuasive aspects of research articles. I explore here how the concepts of reading, interpretation, and criticism can be applied to biology research articles, and I propose specific pedagogical approaches that can be used to exploit these concepts in the classroom.

READING THE EXPERIMENTAL NARRATIVE

The first and most important task in reading a research article is understanding the experimental narrative, the simple description of the methods and results. The experimental narrative is the central informative portion of an article. The conventions of scientific writing demand that authors objectively report what was done and what was found without adding interpretation. Thus, the first task of the reader is to comprehend what was done and what was found, without trying to interpret the findings or assess the conclusions.

Reading the experimental narrative is roughly analogous to what Scholes describes simply as “reading.” Scholes argues that simple reading, even if it is a largely unconscious activity, requires “knowledge of the codes that were operative in the composition of any given text” (Scholes, 1986). In other words, we need to know something about the kind of text we are reading. Thus, students will not fully comprehend research articles unless they understand their structure and conventions. For example, Smith describes upper-level undergraduates struggling with the basics of reading graphs (Smith, 2001). These students may need explicit instruction in concepts such as the difference between independent and dependent variables and in conventions such as plotting independent variables on the x-axis. They may also need to know that they can turn to the methods section if they require information about the techniques used to collect the plotted data. The point is that for students to simply read an article, they need more than mastery of the subject material. They also need to understand the research article “genre” and have a sense for how scientists write. Students may begin to gain this knowledge in laboratory classes, where they are expected to write reports in the form of research articles.

INTERPRETATION

A major rationale for using research articles in courses is to teach students to think like scientists (Levine, 2001; DebBurman, 2002; Mulnix, 2003). Students will begin to achieve this goal when they progress beyond the ability to simply read the experimental narrative and achieve some ability to interpret it. Scholes describes interpretation as putting “text upon text,” as finding meaning or themes within the text. As Scholes points out, this means moving from concrete words, characters, situations, and events, to abstract concepts such as themes and values. In a sense, interpretation hinges upon putting together the individual aspects of a text into a coherent general message.

For scientists, interpretation of research articles involves making sense out of the experimental narrative, for example, by drawing conclusions from a data set, assessing how well experiments test a hypothesis, or determining how conclusively results support or contradict a hypothesis. Authors of research articles almost always advance interpretations of their own results, and this is generally a persuasive activity, especially if the intent is to convince the reader of the validity of a particular conclusion.

The task for students, then, is twofold. Students must not only understand the scientific reasoning of the authors but also develop interpretations that are independent of those of the authors. They must first identify and understand the authors' interpretations. To do so, students need to understand the scientific method and the rules and standards that scientists use when evaluating data. Comprehending the interpretations in a research article can be a challenging activity for undergraduate students. Developing independent interpretations is even more difficult. In my experience, students need practice and experience before they can confidently accomplish these tasks. One method for helping students arrive at independent interpretations is to deny them those of the authors (Smith, 2001). Smith provided students with edited versions of research articles and asked them to interpret the results without access to the authors' conclusions.

Students may wonder why they need to independently develop interpretations of experimental data rather than simply relying on those of the scientists who actually performed the work. The simple answer to this question is that scientists evaluate the primary experimental procedures and results because these form the fundamental currency of scientific argument. Scientists' statements about data, even the highly reliable statements of those who collected the data, are rightly considered to be secondary to the data. By insisting that students focus on data rather than on interpretations of data by other scientists, instructors can emphasize a priority in scientific thinking.

Students are also more motivated to develop their own analysis when they understand that interpretations other than those of the authors may be valid. Scientists may legitimately disagree about interpretations, and readers of research articles may uncover novel interpretations (Muench, 2000). Perhaps new information has become available since the authors wrote the article. Maybe an unexplained discrepancy can be interpreted as an exciting breakthrough if viewed through the lens of new information. Perhaps a different kind of data analysis would illuminate additional relationships within the data. The bottom line is that independent interpretation of data is a crucial aspect of the scientific process. I have found that students become more motivated to evaluate research articles once they learn that independent interpretation is an authentic aspect of an ongoing scientific process, rather than a dry pedagogical exercise.

CRITICISM

Criticism of a research article involves judging its strengths and weaknesses and is obviously persuasive in nature. Although the distinction between interpretation and criticism can be subtle, an important difference is that whereas interpretation involves finding meaning in the experimental narrative, criticism involves connecting the research article to other scientific works. Criticism, according to Scholes, involves making collective judgments about texts, placing “texts against texts.” The term “collective” is important; Scholes argues that criticism is not a matter of personal preference or whim. Instead, criticism can only be performed on behalf of a particular group or worldview. Therefore, criticism cannot be grounded only in a text itself; instead, it must be the result of applying some external framework to the text. In science, we seek to teach students to think as members of the scientific community, and we want students to apply the agreed-upon standards and methods of science.

Students may be surprised to learn that research articles contain criticism, especially because the word often carries a negative connotation. However, if they are encouraged to read closely, students will see that criticism is rather common and often positive. For example, authors may point out the strengths of their own work and cite other studies to support their conclusions. They may argue why their methods are appropriate, discuss the implications of their findings, and address possible objections to their work. Furthermore, authors may endorse or attack the work of other scientists.

For students to understand the criticism within a research article, they must not only be familiar with the standards of the general scientific community but also be engaged with the more restricted critical community of a specialized research field. Every research field operates within the general scientific method, but it also has its own particular challenges and idiosyncrasies. For example, an acceptable number of trials in one discipline might be hopelessly expensive or impractical in another. Although these restricted research communities are difficult for students to access, students can step into them by actively reading research articles within a field. In fact, it is probably only by reading other research articles that students can understand the critical comments in an article and begin to make their own authoritative critical statements.

An example will help illustrate why reading within a discipline is a prerequisite to critical analysis. Suppose a student reads in the Methods of an article that nine crickets were studied. Student A, who has not attempted to join the relevant research community and does not wish to become involved in criticism might state, “They used nine crickets.” Student B, also not familiar with the research community but a bit bolder might argue, “They used nine crickets, and that is not enough. They might not be able to discern small differences between the treatments with such a small number of insects.” In contrast, student C, who has read carefully and thoroughly in the field, might be able to write, “They only used nine crickets, an inadequate number given the low cost and easy maintenance of crickets. Jones and colleagues used forty crickets in a similar study, whereas Smith and colleagues measured 52 crickets. Furthermore, the statistical analysis of Johnson and colleagues shows that at least 20 individuals must be measured to discern treatment differences.” Student C's arguments are truly critical because they draw on the scientific community for support.

The notion that criticism is grounded in research communities can help students overcome their reticence about critiquing research articles. Students sometimes ask me, “Who am I to attack this paper?” or “How could my opinion on this paper make any difference?” One answer to these questions is that I am not asking for their opinion, rather I am asking them to act on behalf of the relevant research community. Students may also be more willing to critically evaluate articles when they understand that it is a normal part of the scientific process. Indeed, scientists expect their work to be critiqued. In this context, it is helpful to explain the peer-review process to students so that they understand that published works have already survived such a critique. Students also will benefit from knowing that strong ethical responsibilities come with membership in the community of scientists. Critiques must be fair and even-handed. Students who learn to evaluate research articles are taking an important step into the community of scientists.

PEDAGOGY

In my courses, I have used the concepts of reading, interpretation, and criticism in different ways. Sometimes, I have used these concepts as a guide to my classroom approach, but I have not made them explicit to the students. This approach seems to work well with beginning science students. Layering a new conceptual framework on top of the difficult task of reading a research article could create unnecessary confusion in these students. In other cases, I have found it advantageous to explain interpretation and criticism and to use them explicitly to guide student work. I have found this to be particularly useful with upper-level students, who are often already aware that research articles contain persuasive content. Interestingly, I have also found that nonscience majors can be receptive to thinking about interpretation and criticism in scientific articles, possibly because it enables them to see connections to the work they do in their primary disciplines. Below, I explore specific strategies that use the concepts of reading, interpretation, and criticism to help students read articles critically.

Goal

Teach students to differentiate between the experimental narrative, interpretation, and criticism in a research article. This is the first step toward fully understanding a research article and developing an independent position.

Strategies. Teachers can select short sections of research articles that contain experimental narrative, interpretation, and criticism and ask students to flag sentences that exemplify each of these processes. Some statements will obviously fall into one of these categories, and others may be less clear cut. The goal is to help students see that different levels of analysis coexist in articles. It is OK, perhaps even useful, if students disagree about the proper designation of some statements. Narrative, interpretation, and criticism are somewhat arbitrary categories placed onto a continuum of analytical complexity. The importance is not the terms themselves but their utility in making plain the multiple levels of analysis in articles. Instructors can also assign questions that explicitly engage different levels of analysis. For example, a question aimed at simple reading might be, “What is the experimental hypothesis?” A question aimed at interpretation might be, “How well do the data support the hypothesis?” Finally, a question aimed at criticism might be, “How does the conclusion of this study compare with the findings of a related paper?” Of course, many instructors already ask questions like these (Levine, 2001); my suggestion is that it is helpful if the different levels of analysis inherent in such questions are understood by instructors and students.

Goal

Teach students to develop interpretations and criticisms that are separate from those of the authors. This is the basis of taking a critical stance toward a research article.

Strategies. Instructors can model strategies that experienced readers use to maintain critical distance from the author, for example, the strategy of selective or nonlinear reading (Bazerman, 1988; Charney, 1993; Pall, 2000). Critical readers know that the experimental narrative is to be found in methods and results and that these sections are mostly free from authors' interpretations and criticisms. Readers who seek to maintain an independent stance will focus on these sections. In fact, some readers may scrutinize graphs and tables before reading the text of the results section, with the intention of making their own judgments about the data before reading any statement about them. Approaching a research article this way in class can help students see how to maintain a critical stance. Attention to how we phrase our questions might also help students differentiate between their interpretations and those of others. For example, instead of asking, “How well do the data support the hypothesis?”, we might ask two questions: “What is your interpretation of the authors' data?” and “How does your interpretation compare with that of the authors?”

Goal

Teach students to enter the critical community of scientists, a prerequisite to making an authoritative criticism of a research article. This is a challenging task for undergraduate students, because it demands a familiarity with the primary literature of a research field.

Strategies. Teachers can assign “paired articles.” Students will not have the time or inclination to read the primary literature extensively for each assigned research article, especially if we hope to use them regularly. Thus, we might start by assigning paired articles, two research articles that somehow speak to each other. They might come to contradictory conclusions or to the same conclusion through different approaches. Two articles are the minimum unit required for a critical discussion, and students can begin to apply “text against text” in this setting. Again, the form of our questions about an article pair will be important. We might ask, “Does one article criticize the work of the other?” and then “What is your assessment of this criticism?” Or we might ask students to place themselves in the position of the authors of the criticized work, “How would you respond to these criticisms?”

SUMMARY

Students sometimes struggle to develop critical analyses of research articles. Part of the difficulty for students is challenging subject material, but I contend here that another hurdle for students is the persuasive aspect of research articles. The framework of reading, interpretation, and criticism is one tool that instructors can use to help students confidently evaluate the persuasive features of research articles.