Bioliteracy and Teaching Efficacy: What Biologists Can Learn from Physicists

Abstract

The introduction of the Force Concept Inventory (FCI) by David Hestenes and colleagues in 1992 produced a remarkable impact within the community of physics teachers. An instrument to measure student comprehension of the Newtonian concept of force, the FCI demonstrates that active learning leads to far superior student conceptual learning than didactic lectures. Compared to a working knowledge of physics, biological literacy and illiteracy have an even more direct, dramatic, and personal impact. They shape public research and reproductive health policies, the acceptance or rejection of technological advances, such as vaccinations, genetically modified foods and gene therapies, and, on the personal front, the reasoned evaluation of product claims and lifestyle choices. While many students take biology courses at both the secondary and the college levels, there is little in the way of reliable and valid assessment of the effectiveness of biological education. This lack has important consequences in terms of general bioliteracy and, in turn, for our society. Here we describe the beginning of a community effort to define what a bioliterate person needs to know and to develop, validate, and disseminate a tiered series of instruments collectively known as the Biology Concept Inventory (BCI), which accurately measures student comprehension of concepts in introductory, genetic, molecular, cell, and developmental biology. The BCI should serve as a lever for moving our current educational system in a direction that delivers a deeper conceptual understanding of the fundamental ideas upon which biology and biomedical sciences are based.

INTRODUCTION

In today's world, we are faced with amazing scientific advances. To zoom into deep space through the Hubble Telescope's eye1 provides a perspective equal to the more well-known image of the earth as a pale blue dot.2 Similar explosive increases in our understanding and technical abilities are occurring within the biological sciences. In contrast to advances in physics and astronomy, biological discoveries often directly impact daily personal, social, and political decisions. One study announces that a certain behavior will increase the possibility of contracting disease A by N%, while another suggests that genetic factors are more important. A small, and perhaps practically insignificant, increase in the rate of a particular disease may be used to justify the expense of billions of dollars that could be used to better effect, that is, save more lives, if invested elsewhere. Even shopping for groceries presents us with a test of our biological savvy. Is it really necessary to supplement cooked forms with enzymes? Do genetically engineered variations of familiar fruits and vegetables pose a danger, even if they are more nutritionally balanced and may be contaminated with lower levels of natural toxins? Are herbal and “nutraceutical” supplements capable of delivering on promises beyond those attainable by the most expensive and rigorously tested pharmaceutical? Are their potential health risks as clearly stated as their potential benefits?

The general public is very much aware of the fast-paced advances being made in the biological sciences. Biological and biomedical breakthroughs are disseminated in both the professional and the popular press. Their ramifications can be widespread, even if later they turn out to be incomplete, incorrect, or nonreproducible. Unbalanced and generally uncritical publicity can lead to unrealistic expectations and demands, demands often met, unfortunately, by unscrupulous charlatans.

While the realms of physics and astronomy appeal to our intangible sense of wonder, we are more directly aware of issues related to life, death, sickness, and health. It does not follow, however, that the general public's understanding of even the most fundamental principles of biology is better than its understanding of physical principles. Much of this lack of understanding can be directly tied to poor teaching and learning at all levels of our educational system. Our current system perpetuates rote, rather than meaningful learning (Novak and Gowin, 1984; Novak, 2002). In effect, students are capable of sounding out the words in the sentences, but not of comprehending their meaning. Just as reading literacy combines the abilities to comprehend and interpret words on a page (fluency), bioliteracy requires the ability to do more than simply list and label—it requires conceptual understanding, the ability to transfer knowledge and understanding to other domains. In this article we seek to make other educators and biologists aware that our team is working to develop biology concept inventories in the areas of introductory, genetics, molecular, cellular, and developmental biology, a program still in its early phases. Our goal is to enlist their feedback and assistance in this community endeavor, the developmental of a series of instruments that will be useful nationally.

Studies of the U.S. population's understanding of basic biological processes and concepts reveal a rather alarming level of both ignorance and misunderstanding (National Science Foundation [NSF], 2002). Shamos (1995) argues that the general public's overall level of scientific literacy has not improved in a century. The public's profound ignorance of, and often misinformation about, matters biologic is illustrated by patient demand, and physician prescription, of antibiotics to treat viral infections,3 the billion-dollar-a-year market in ineffective nutraceuticals4 and “alternative” therapies,5 and the socially and medically irresponsible rejection of childhood vaccination6 (Friedlander, 2001; Wanjeck, 2002).

The American Association for the Advancement of Science's (1985) Project 2061 began in 1985 with an eye toward making all Americans scientifically literate. Its goals are to promote familiarity with the natural world, an understanding of key concepts and principals, and an ability to think in scientific ways and to recognize the connections between and among the various sciences. As opposed to a stale catalog of facts, Project 2061 aims to convey the reality of science as an exciting and creative human enterprise. Science literacy implies an ability to apply science knowledge to personal and social phenomena.7 While the Project 2061 “Benchmarks for Science Literacy”8 does identify specific areas for conceptual competence in the life sciences, it also remains general in the sense that it focuses on traditional concepts in life science. Similarly, the Biological Sciences Curriculum Study (BSCS) description of literacy is divided into three levels that are specific to biology. While broad-reaching, they do not provide details aimed at particular concepts.

Bioliteracy implies conceptual understanding. A bioliterate person not only comprehends scientific terms, but has the ability and confidence to apply knowledge learned in one setting to another and to make informed judgments about new discoveries based on a solid understanding of fundamental principles (e.g., Bloom et al., 1956). Thus, bioliteracy includes a working knowledge of scientific method and practice. We choose to define bioliteracy in this way because our experience as educators has demonstrated that conceptual understanding is the key to meeting more general science literacy criteria, e.g., state standards and those from Science for All Americans.7 To measure bioliteracy, therefore, we must measure conceptual understanding. Unfortunately, there is currently no general, well-designed assessment “instrument” available to instructors for identifying gaps in student understanding upon leaving high school and entering college or, for that matter, upon the completion of a college degree in biological science. Those standardized tests that do exist, such as the Biology Advanced Placement Exam, are designed more to distinguish among students than to assess student understanding of fundamental concepts. A quick review of some of the questions on the Biology Advanced Placement Exam reveals materials, such as the order of enzymes in the tricitric acid cycle, that capture no basic understanding of fundamental biological principles and concepts. Rather, it privileges the retention or rote learning of molecular or terminological trivia. As such, such exams are not a useful measure of bioliteracy, which hinges on conceptual understanding. We must turn to other fields for examples of standardized tests that examine scientific literacy in the way we use the term here: as a means of assessing student fluency in a subject area and as a measuring stick of and impetus for curricular reform aimed at improved teaching and learning.

THE FORCE CONCEPT INVENTORY (FCI) AND ITS IMPACT ON PHYSICS TEACHING

The Newtonian concept of force is in many ways counterintuitive to our day-to-day experiences. This fact makes teaching Newtonian physics difficult. To assess whether students have mastered this counterintuitive set of ideas, David Hestenes and colleagues (1992) developed and validated the FCI. The FCI provides a standard measure that makes it possible to compare the relative success of different teaching strategies. (The Web site Field-Tested Learning Assessment Guide has general instructions on such test development and application.)9 The results clearly demonstrated that standard lecture-based instruction was not sufficient to bring most students to a level of concept mastery, independent of the lecturer. A comparison of 6,000 students' scores on the FCI, taken from a variety of types of postsecondary education programs and across teaching approaches (from high levels of active learning to traditional lecture), shows that both inspired and engaging lecturers as well as the less gifted, less motivated or motivating lecturers attained the same level of student conceptual understanding (Hake, 1998). It was possible to conduct this broad-reaching metastudy only because the FCI is a widely accepted and used measure of student conceptual understanding in introductory physics. Though counterintuitive, such results have appeared in the psychological literature for many years, with the overall conclusion that “only lecture” reinforces “only memorization” (McKeachie et al., 1990; Bligh, 2000).

At the University of Colorado, Boulder, at the University of New Mexico, and around the country, the ability to assess learning outcomes objectively has prodded many physics and astronomy instructors to take a new look at how they have been teaching. Exemplified by Eric Mazur's (1997) work at Harvard University on active learning in large lecture classrooms, this effort has led to the implementation of a variety of alternative teaching approaches, the successes of which have now been well documented. A recent guide is available in astronomy (Green, 2003) and a few textbooks incorporate this teaching transformation (Zeilik, 2002). Yet what is central to this emerging teaching revolution is the ability to assess students' conceptual learning in a standardized way. Without an appropriate instrument with which to measure conceptual understanding in our students, educational experiments can lead to unfounded conclusions and self-delusion, particularly in the instructors who initiate them.

Physics education research has advanced to the point that a synthesis has been achieved, at least for introductory, calculus-based courses. Redish (2003) has brought together research and practice in physics and Adams and Slater (2003) have created a similar guide for astronomy. We expect that a similar reassement of curricular materials and teaching approaches will emerge from the application of a biological concept inventory.

A BIOLOGICAL CONCEPT INVENTORY (BCI)

There have been a number of previous efforts to identify misconceptions in biology education. This literature tends to fall into one of three categories: (1) teaching approaches and interventions designed to address student misconceptions (e.g., Amir and Tamir, 1994; Sanger et al., 2000; Wyn and Stegink, 2000); (2) work examining textbooks and other potential sources of misconceptions (e.g., Odom, 1993; Story, 1989, 1990, 1991, 1992a, 1992b); and (3) work focused on bringing techniques such as concept mapping to teachers so that they can use them effectively in their classrooms (Ault et al., 1984; Browning and Lehmann, 1988; Fisher et al., 2000). Based on the results of this long-standing research, development, and testing effort, teachers are urged to identify and confront misconceptions with their students so that they can replace the student's private mental models with understanding grounded in what is accepted in the scientific community (e.g., Committee on Undergraduate Science Education, National Research Council, 1997). We look to these studies to assist us in identifying biological concepts that students tend to misapprehend and to reveal the types of mistakes they make.

What this literature does not provide us with are (1) a comprehensive and coherent concept base for biological science and (2) adequately validated concept inventories that cover these concepts. Existing instruments, such as the recently released Conceptual Inventory of Natural Selection (Anderson et al., 2002), provide test items and distracters for a topic not explicitly covered in our program. Others, such as the Diffusion and Osmosis Diagnostic Test (Odom and Barrow, 1995), have been shown to have questionable reliability (see Griffard and Wandersee, 2001), at least in terms of their ability to identify students' misconceptions accurately based on their naive understanding of concepts rather than their factual and vernacular misconceptions.

Why has an instrument with the same potential to impact teaching and learning enjoyed by the FCI not appeared in the biological sciences? One reason may be the belief among many that biology and physics face different pedagogical hurdles, that there is something qualitatively different between the concepts the two disciplines seek to convey. The reality is that biology, like Newtonian mechanics, is littered with misconceptions derived from students' “everyday” experiences (e.g., Browning and Lehmann, 1988; Brumby, 1979; Fisher and Moody, 2000). The notion that mushrooms and humans share the same basic molecular mechanisms is certainly not a common-sense conclusion. That tens of thousands of genes, rather than “blood,” pass essentially unaltered from generation to generation is not obvious. That organisms obey the laws of thermodynamics, that errors (mutations) and selection lead to the accumulation of information and evolutionary novelty, and that mutations can be beneficial in one environmental context (or stage of life) and detrimental in another are all ideas central to understanding the workings of life and serve as the conceptual foundations for more detailed and abstract processes. Students have direct, everyday experiences with heredity, health, and disease. They have internalized naive explanations for many of their “observations” and have incorporated these ideas into their sense-making process. Misunderstanding fundamental biological concepts reinforces and fosters a plethora of ideologies and beliefs that can lead to poor personal decision making and public policy. As an obvious example, the fact that many political leaders in the United States can seriously maintain that the “jury is still out” on the validity of the theory of evolution emphasizes the failure of the biology teaching community to address common misconceptions directly and effectively!

Recently, coauthor Michael Zeilik spoke with biologists at professional meetings about developing a BCI. The most common response was that “we should have one,” followed by “it can't be done” because of a lack of consensus about the conceptual content and level of “Biology 101.” This same objection was used for years by astronomers with regard to designing a concept inventory for “Astronomy 101.” When a team finally did form to develop an Astronomy Diagnostic Test (ADT version 2 [see Deming, 2002; Hufnagel, 2002; Zeilik, 2003]), they attacked this issue directly by selecting particular concepts grounded in national standards as well as teaching practices. First, they examined the astronomy content of the National Science Education Standards (NSES, NAS) and restricted ADT questions to this K–12 content. Second, instructors for the National Survey were requested to fill out a content grid that mapped their course to ADT items. Clearly, if a content area was not “covered” in the course, a pre/post change would not be expected. Third, they emphasized to users that they should examine the average of the whole ADT rather than any one item as a measure of conceptual gain. Fourth, they carefully developed the ADT to have both acceptable reliability and acceptable validity, from a psychometrics view, as well as that of an astronomer! In contrast, choosing the concepts to be covered by the FCI in physics was relatively easy because the authors chose to focus on the agreed-upon content for a portion, but not all, of the standard introductory physics course. The team selecting the concepts for the ADT, however, faced many of the same challenges and lack of consensus about the content essential to introductory astronomy that the developers of a BCI face, and we plan to adopt a similar approach in the development of our series of concept inventories.

Where do we begin with the construction of a useful BCI? In our minds, there is a series of concept “types” that need to be addressed. The first is the introductory level of bioliteracy at the end of high school that we can reasonably expect our secondary education system to provide the students, both as citizens and as our undergraduates. While we do not wish to reinvent the wheel, we feel that a basic concept inventory that provides multiple choice-style questions about the nature of science, the thermodynamic properties of life, evolutionary processes and the molecular basics of heredity and cellular organization is needed at this level. An obvious place to start is with previously established standards, such as the Biol 2010 recommendations published by the NAS as well as Project 2061's life science and evolution standards.

Clearly, what counts as a complete biology concept inventory is open to debate. We are currently working most actively on BCIs in two areas: basic bioliteracy and developmental biology. The basic BCI is intended as a measure of what our secondary education system should be expected to produce and what literate citizens will need to know to make informed biology-based decisions. The second—the developmental BCI—is intended as a tool to measure learning efficacy in standard and “transformed” versions of our major's developmental biology course.10 As these are completed, our intent is to develop BCIs focused more tightly on the areas of genetics, cell biology, and molecular biology, followed by the development of BCIs addressing the areas of ecology and physiology. These advanced BCIs are designed to assess concept fluency at the college level and will be used to provide us with the pre/post learning assessments that are an essential element for determining the degree to which a course innovation is working, as well as a means for comparing before and after course transformation learning.

A CUSTOMIZABLE, MULTIDIMENSIONAL BCI

At present we are working on a format that can be used to cross-correlate what instructors think they are teaching with what students are learning, thus providing an important feedback system (Figure 1). This format provides a perspective on student learning gains in relationship to the importance and emphasis placed on the concept by the instructor. It is one thing to have students “miss” a concept that was briefly mentioned in passing and another when a good number of them miss something you felt you really spent time on during class. This format also provides a means of gauging how well a particular teaching innovation worked. If all of the concepts covered through classroom collaboration activities show little or no learning gains, then there is a fundamental problem with the way collaboration was implemented in the course, or the approach was not appropriate for the concept. Please note that Figure 1 is not a question worded for a concept inventory, but simply an example of what the format of the inventory will look like when an instructor accesses it on-line.

Figure 1.

An example of an interactive question in which instructors indicate what concepts they teach and the importance they attach to specific concepts in their classes.

While multiple-choice tests are unambiguous to grade and meet issues of scale and objectivity, they have their own drawbacks. Students can adopt testing strategies that enable them to answer the questions reasonably successfully with little solid understanding of the material covered (e.g., Schneps, 1994). Yet, at the same time we must acknowledge that we face real constraints in terms of available energy and resources needed for grading more complex answers. Much as we all realize that concept maps, open-ended responses, essay questions, and even lab tests provide greater insight into student knowledge, they take too many resources (in terms of time and energy) to grade rigorously and objectively. We can, however, improve the standard multiple-choice test through the use of what is known as the two-tiered or two-dimensional test (Treagust, 1988). In this method, students are asked questions that offer known misconceptions as distracters and are asked for their degree of confidence in their answer (Figure 2). In an unscored model, particularly useful in the learning rather than the evaluative setting, this method allows us to identify particular students' misconceptions as well as their level(s) of confidence in their knowledge. To be most useful, this information should be readily available to instructors so as to enable them to customize their in-class work to address those specific issues that students find most difficult.11 Also, it is important to be aware that confidence measures have, in many contexts, been shown to be gender-biased—it is therefore particularly important that validity and reliability measures be employed during development.

Figure 2.

An example of a two-dimensional test question (again, meant to demonstrate format rather than actual questions as they will appear in the BCI). In a graded model, the maximum reward is associated with a correct and confident answer and a penalty is associated with incorrect and confident answers.

CONCLUSIONS

Our goal is to improve bioliteracy among students, both for those leaving the secondary school level and for those who major in biology at the college level. In order to know what the current level of bioliteracy is, to document improvements in student understanding of fundamental concepts in the field and its related subdisciplines, and to determine which teaching approaches lead to higher levels of student learning, we need to have a reliable instrument by which to measure student comprehension and capabilities, not only in recognizing the technical meanings of specific words and concepts but in being able to apply them correctly. Our effort seeks to map the domain of biological literacy beyond rote learning: the conceptual understanding that provides the foundation for bioliteracy. Literacy is more than knowing the mechanics, e.g., the facts, the letters, the individual words; it assumes fluency. It is fluency that separates literate persons from those who can only mouth the words and do not understand their meaning or how to put them together to express their own ideas or to describe and explain phenomena in their world. To measure bioliteracy, we must be able to determine students' levels of conceptual understanding and fluency. We need instruments that do more than test rote learning; they must assess meaningful learning (e.g., Bransford et al., 2000; Novak, 2002).

Past efforts at assessing conceptual understanding in the biological sciences have not provided the breadth to help guide teaching and learning. Given the powerful impetus that concept inventories have created for improved teaching and learning in the fields of physics (the FCI) and astronomy (the ADT), it seems urgent that we develop similar instruments for biology. We intend to capitalize on the lessons they learned as we develop our own procedures (Figure 3). We will begin by identifying concepts through examination of national standards, our teaching experiences, and interviews with students. This information will also be used to generate distracters in the inventory. We will pay particular attention to the language students use to express their understanding. “Beta” versions of questions and distracters and student “think alouds” will be used to look for validity: Are students “hearing” the same question you intended to ask? Are they picking responses based on misconceptions rather than simple ignorance of the words used? Are the distracters worded in ways that enable students to interpret them as developers intended. We will revise the BCI questions and distracters and pilot the instrument through a number of cycles to ascertain both its validity and its reliability for a wide range of student populations.

Figure 3.

Question 1 shows a traditional physics problem, while question 2 illustrates a conceptual problem that uses well-documented misconceptions as distracters (Mazur, 1997, p. 5). We have not yet progressed to the point that we have research-based and validated concept questions for biology, but we offer an example here. Question 3 has the form of a concept inventory question, whereas question 4 is a more typical multiple-choice question. This sort of question would appear as part of a domain-specific BCI rather than a basic BCI. Currently, it is worded in the language of professors of biology rather than that of students. Through the interview process, we would identify the language we need to use in the question and in the candidate responses that captures students' conceptual understanding rather than their ability to give a learned response.

Our goal is to develop a series of instruments that transcend the needs of our research-intensive university; instruments that address the needs of the broad range of secondary and postsecondary institutions. To do this, we have chosen to publish this article early in the development phase so that we can involve others in the biological community in the formulation of the key concept statements (lists in each subject area that are the foundation of a valid BCI). Our aim here is not to introduce readers to a finished product but to engage interest and participation in the development of the BCI among research biologists and educators interested in assessment measures that are scientifically grounded, reliable, valid, and convenient to administer. We invite you to log on to our Web site at http://bioliteracy.net to rate the concepts presented (essential, important, marginal) and to suggest new concepts that should be included in the areas of introductory (end of high school/beginning college level), molecular, cellular, and developmental biology and genetics. We also invite you to share common misconceptions that you have encountered regarding these concepts.

Can we learn from physics and astronomy when it comes to determining the level of bioliteracy among our students? Yes, we can. Will we succeed in building the multiple instruments we envision for the BCI? Yes, we will—after all, while we are not rocket scientists, we are biologists.