Investigating Undergraduates’ Perceptions of Science in Courses Taught Using the CREATE Strategy

Abstract

Many science educators agree that 21^st century students need to develop mature scientific thinking skills. Unsurprisingly, students’ and experts’ perceptions about the nature of scientific knowledge differ. Moreover, students’ naïve and entrenched epistemologies can preclude their development toward “thinking like scientists.” Novel teaching approaches that guide students toward more mature perceptions may be needed to support their development of scientific thinking skills. To address such issues, physics educators developed the Colorado Learning Attitudes About Science Survey (CLASS), subsequently adapted for chemistry and biology. These surveys are “designed to compare novice and expert perceptions about the content and structure of a specific discipline; the source of knowledge about that discipline, including connection of the discipline to the real world; and problem-solving approaches” (Semsar et al., CBE Life Sci. Educ. 10:268–278; p 269). We used CLASS-Bio to track students’ perceptions of science in separate first-year and upper-level CREATE (Consider, Read, Elucidate hypotheses, Analyze and interpret the data, Think of the next Experiment) electives, hypothesizing that perceptions would become significantly more expert-like across a semester. Both first-year and upper-level cohorts made significant expert-like shifts. Students also made significant critical thinking gains in CREATE courses. Our findings of more mature, expert-like perceptions of science post-course contrast with those of previous studies, where students’ thinking became significantly less expert-like across a term of introductory instruction and changed little in upper-level biology electives. Augmenting traditional biology curricula with CREATE courses could be an economical way to help undergraduates develop more mature views of science.

INTRODUCTION

We are interested in the influence of pedagogy on maturation of undergraduates’ perceptions of science, as the maturity of students’ views can influence their ability to think scientifically (1). Teaching approaches that emphasize student engagement (2, 3) have produced outcomes in alignment with workforce demand for increased numbers of STEM (science, technology, engineering, and mathematics) graduates (4). Less well-understood, however are questions of how students’ deeper scientific thinking, epistemological views of science, or understanding of the nature of science become more sophisticated or expert-like. As noted by D. Kuhn, “[s]cientific thinking tends to be compartmentalized, viewed as relevant and accessible only to the narrow segment of the population who pursue scientific careers. If science education is to be successful, it is essential to counter this view, establishing the place that scientific thinking has in the lives of all students” (5). In essence, scientific thinking should not be seen as the exclusive province of “experts.” Ideally, educators would guide undergraduates toward developing more mature perceptions of science and increased scientific thinking ability, along with subject-matter expertise.

We hypothesized that single-semester CREATE courses would promote maturation of students’ scientific perceptions and scientific thinking. As developed in earlier work, courses taught through the CREATE strategy couple intensive analysis of scientific literature with e-mail interviews of papers’ authors to demystify and humanize science (6). CREATE courses are not based on textbooks, lectures, or PowerPoint presentations. Instead, primary and secondary scientific literature is the focus of both pre-class homework assignments and in-class activities. Using a “toolkit” that includes concept mapping, sketching, annotation of figures, transforming data tables into graphs/charts, and/or paraphrasing jargon-rich sentences (7–9), students compile homework and background information, sourcing much of the latter independently, in portfolios brought to each class meeting. In class, students are guided to consider how/why a study was designed as it was, and to analyze outcomes as if they had performed the reported studies themselves. Discussions focus on close analysis of each figure and table, with substantial related content information reviewed as a basis for understanding the studies performed and data presented. Work in small groups alternates with whole-class discussions and debates. Experimental design and grant-panel activities stimulate student creativity while developing collegial argumentation skills. Testing is open-portfolio, relieving students of the pressure to memorize and underscoring CREATE courses’ focus on understanding/explaining data and thinking of creative ways to extend the studies examined. E-mail surveys of the papers’ authors provide behind-the-scenes insights and illuminate the diversity of individuals who become scientists (8).

We investigated the question of students’ perceptions of science, naïve vs. expert-like, using the biology version of the Colorado Learning Attitudes About Science Survey (CLASS-Bio; see Appendix 1 for further information on development of this survey by Semsar et al. (1)). We used CLASS-Bio in two different semester-long CREATE electives at the City College of New York (CCNY), one for first-year students and a separate course for upper-level students. The CCNY first-year CREATE course has been described (10); it focuses on scientific literature chosen to emphasize scientific thinking. The upper-level elective has also been described (6–8); it is narrowly-focused on a series of primary literature articles tracking a particular scientific investigation as it developed over a period of years.

Previously, we found that CCNY students made diverse cognitive (e.g., critical thinking, experimental design) and affective (e.g., self-rated attitudes, epistemological maturation) gains in first-year (10) and in upper-level (6, 9) CREATE courses. We used the CLASS-Bio survey instrument in the present study to examine students’ perceptions of science and of scientific thinking across the semester in two different CREATE electives designed for distinct cohorts of students. Our data reflect three cohorts of Biology 10050, Introduction to Scientific Thinking, a CREATE course for STEM-interested first-year students, and four cohorts of Biology 35500, Analysis of Scientific Literature Using CREATE; designed for third- or fourth-year students. As outlined by the CLASS-Bio developers, experts and novices differ in their perceptions about: “1) content and structure of knowledge, 2) source of knowledge, and 3) problem-solving approaches” in biology (1; p. 269). Per an extensive development process (Appendix 1), CLASS-Bio addresses perceptions of such issues and can be used with biology undergraduates “to characterize how student perceptions match expert perceptions about how knowledge is structured, where knowledge comes from, and how new knowledge is gained and problems are solved in the field of biology. … Furthermore, this instrument can be used to evaluate impacts of pedagogical reform on students’ growth of expertise in scientific thinking” (1; p. 277).

We administered the CLASS-Bio survey pre- and post-course in two different CREATE electives taught by coauthor SH. As curricular restraints on our campus preclude running two electives in parallel and using one as a control section, we discuss our findings in relation to published accounts from other campuses that used CLASS-Bio in: (a) semester-long Introductory Biology courses (1); (b) upper-level electives (1); (c) Introductory Biology for majors or for nonmajors (11, 12); and (d) in a longitudinal study (13).

METHODS

Study design

CREATE courses were taught at two levels, first-year and junior/senior, in two different electives at CCNY and assessed using CLASS-Bio to test whether CREATE students’ perceptions of science would shift in the expert-like direction across the semester. Such shifts would reflect maturation of understanding of the nature of biological knowledge and increased scientific thinking skill (1). The study includes four iterations of the upper-level course (Biology 35500; biology and chemistry/biochemistry majors) and three iterations of the first-year elective CREATE course (Biology 10050; STEM-interested; majors not yet declared). All courses were taught by one of the authors (SH). As an independent measure of students’ thinking, we used the Critical-thinking Assessment Test (CAT; 14, 15), which we had previously administered in the first-year CREATE course (10). We used the CAT in this study with one cohort of upper-level (Bio 35500) CREATE students. Financial and time constraints precluded additional use of the CAT.

Student participants in the first-year CREATE elective, Introduction to Scientific Thinking, are drawn from freshmen whose math placement scores necessitate additional mathematics coursework. The first-year CREATE course (Biology 10050) is typically taken in the first college semester and has no prerequisite beyond an interest in STEM. The course meets for 75 minutes twice a week, usually enrolling 15 students.

Analysis of Scientific Literature Using CREATE, the upper-level (juniors/seniors) course, is an elective meeting twice a week for 100 minutes per session, enrolling 25 to 30 students. Upper-level students self-select into the Biology 35500 course, which is one option among multiple electives, with a prerequisite of Introduction to Genetics and a pre- or co-requisite of Cellular/Molecular Biology.

Assessment methods

Participant recruitment

In both CREATE courses, at the first class meeting of the 14-week semester, students were invited to participate anonymously and voluntarily in a science education study aimed at “improving undergraduate science education.” Participants were required to be 18 or over, thus limiting participation of some of the first-year students. Participants received no participation or extra-credit points. Nonparticipating students were provided course-related material to read during the survey time. None of the upper-level students had previously taken the first-year CREATE course.

Surveys and survey administration

The CLASS-Bio survey instrument (1) consists of 32 statements (e.g., “For me, biology is primarily about learning known facts as opposed to investigating the unknown”) to which students respond on a five-point Likert-style scale. An overall score reflects pooled responses to all survey statements. Seven individual categories comprise subsets of statements and address specific aspects of scientific thinking and perceptions of science: Real world connection; Enjoyment; Problem solving (PS), reasoning; PS, synthesis and application; PS, strategies; PS, effort; and Conceptual connections, memorization. Surveys were distributed as paper/pencil handouts in the first (pre-) and final (post-) class sessions and took 10 to 15 minutes to complete. See Appendix 1 for a brief review of CLASS-Bio development by Semsar et al. (1).

In one section of the upper-level elective, we also administered the Critical-thinking Assessment Test (CAT; 14), an essay-format instrument that has been validated for undergraduate populations (14, 15). The one-hour closed-book test challenges students to read information from multiple sources (in areas of science unrelated to CREATE course material), interpret associated data, draw conclusions, and write responses to 15 open-ended prompts. The tests were given pre- and post-course, then scored and analyzed by a trained and experienced team at Tennessee Technological University, where the survey was created. Scoring by the CAT team has been documented as highly reliable (15; Cohen’s d > 0.9; 16).

Participating students invented and used personal code numbers on all surveys, allowing pairing of pre- and post-course surveys for statistical analysis without compromising anonymity (6). The matched-pair data represent a subset of total student participants, due to absences on some survey days. For CLASS-Bio, we present both matched-pair data (comparing paired responses from individual students, pre- and post-, thus same N for pre- and post- cohort) and all-participant data (comparing all “pre-” responses with all “post-” responses, different N for each group due to some students’ absences on either the pre- or post-survey day). Outcomes for the CAT are all based on matched-pair data (N = 15 chosen at random by the CAT scoring team from the pool of surveys sent for scoring (15)). The project was deemed exempt (categories 1 and 2) by CUNY IRB (HRPP # 2015-0685).

Survey scoring

We used the student-derived codes to pair pre- and post-course surveys from the same anonymous individual (e.g., 12569 pre- and 12569 post- were a “matched pair” of surveys; see (6)). Matched-pair data have equal pre- and post-course N values. Because individual students may miss either the pre-course or post-course surveys, we also analyzed the larger pool of “all-participant” data that encompasses the matched pair cohort combined with the “students who took the pre-course survey only” and “students who took the post-course survey only” cohorts. All-participant data thus have different pre-course and post-course N values.

CLASS-Bio

CLASS-Bio surveys were scored as described in Semsar et al. (1). The survey developers interviewed a diverse group of Biology PhD holders during the development process and consolidated these individuals’ views into the “expert” perceptions. Per (1), we collapsed the five-point scale into a three-point scale (combining responses 4 and 5 as “agree” and 1 and 2 as “disagree”); and we tracked the percentage of statements on which students’ responses matched those of “experts,” both Overall (all statements) and in the seven categories. CLASS-Bio scores reflect the percentage of total student responses that matched experts’ responses on the same survey statements.

CAT

As defined by the CAT development team, each of the 15 CAT questions addresses several sub-skills: evaluation and interpretation of information (E/I); problem solving (PS); creative thinking (CT); and/or effective communication (EC; 15). The CAT team at Tennessee Technological University provided an analysis of students’ score outcomes, including pre-/post- class averages on the full test and for each sub-skill as well as statistical analysis by paired t test.

Statistical analyses

The majority of the studies with which we wished to compare outcomes used t tests with a criterion of p < 0.05 to determine significance; we analyzed our data using the same test. For matched-pair data, we used a two-tailed matched-pairs t test; for analyses of all-participant data, a two-tailed t test for unequal samples; calculations were done in Microsoft Excel. We also calculated effect sizes (Cohen’s d; 16). Normalized learning gains (LGs) were calculated as the ratio of classes’ average change in score (post % - pre %) and the maximum attainable average change (100% - pre %; 17). We received class average outcomes on the CAT surveys (15) and used these to compute class average normalized learning gains.

RESULTS

Significant shifts on CLASS-Bio in CCNY CREATE courses

The pooled outcomes of three first-year matched-pair CREATE cohorts and (separately) four upper-level matched-pair CREATE cohorts show significant expert-like shifts in multiple CLASS-Bio categories as well as Overall (p < 0.05; 2-tailed matched-pair t tests; Figs. 1A, 1B). The first-year matched-pair students (Biology 10050; N = 21 matched-pairs; Table S1a, Appendix 2) made significant shifts in the direction of expert-like perceptions in two Problem Solving categories—PS, synthesis and application, and PS, effort—and in three additional categories—Real world connection; Enjoyment; and Conceptual connections, memorization (Fig. 1A, Table 1A). First-year all-participant (N = 39 pre-; 30 post-) outcomes are similar, with significant expert-like shifts Overall, and in three PS categories: synthesis and application; effort; strategies. Effect sizes ranged from 0.45 to 0.81 (matched-pairs); and 0.53 to 0.69 (all-participants; Table S1a, Appendix 2).

FIGURE 1A.

First-year CREATE students’ CLASS-Bio outcomes across one semester. CLASS-Bio outcomes in three iterations of the first-year Introduction to Scientific Thinking elective. First-year CREATE students made significant shifts on CLASS-Bio. Students took the CLASS-Bio survey in course weeks 1 and 14. Scoring as described in (1); percentages are the percentage of student responses that matched experts’ responses. CLASS-Bio tracks an Overall score and seven category scores. First-year CREATE students made significant shifts in the direction of expert-like thinking both Overall and in 5 of 7 CLASS-Bio categories. Significance, ** = p < 0.01; * = p < 0.05; matched-pair 2-tailed t test. Error bars, ± SEM. N = 21 matched-pairs; see Table S1a, Appendix 2, for p values and effect sizes for matched-pairs as well as for entire first-year cohort (all-participants)

FIGURE 1B.

Upper-level CREATE students’ CLASS-Bio outcomes across one semester. Upper-level CREATE students made significant shifts on CLASS-Bio. CLASS-Bio outcomes in four iterations of the upper-level “Analysis of Scientific Literature using CREATE” elective. Students took the survey in course weeks 1 and 14. Scoring as described in (1); percentages are the percentage of student responses that matched experts’ responses. CLASS-Bio tracks an Overall score and seven category scores. Upper-level CREATE students made significant shifts in the direction of expert-like thinking both Overall and in 4 of 7 CLASS-Bio categories. Significance: *** = p < 0.001; ** = p < 0.01; matched-pair 2-tailed t test. Error bars, ± SEM. N = 56 matched-pairs; See Table S1b, Appendix 2 for p values and effect sizes for matched-pairs as well as for entire upper-level cohort (all-participants).

TABLE 1A.

CLASS-Bio outcomes in first-year CREATE and in introductory courses on other campuses.

CLASS-Bio category	First-year CREATE (CCNY)			Introductory Biology Aa			Introductory Biology Cb			Introductory Biology (majors)c			Introductory Biology (nonmajors)d
	% agree with experts		Nature of signif shift	% agree with experts		Nature of signif shift	% agree with experts		Nature of signif shift	% agree with experts		Nature of signif shift	% agree with experts		Nature of signif shift
	pre	post		pre	post		pre	post		pre	post		pre	post
Overall	67.7	77.3	Expert	59	57	Novice	68	59	Novice	57.8	34.3	Novice	45.5	46.8	—
Real world conn.	73.9	84.8	Expert	68	66	—	80	68	Novice	65.3	35.9	Novice	53.2	52.9	—
Enjoyment	69.3	82.3	Expert	47	46	—	73	63	Novice	48.6	42.8	Novice	28.1	31.5	Expert
PS, reasoning	75.2	86.7	—	73	69	Novice	82	70	Novice	69.1	36.4	Novice	60.2	53.2	Novice
PS, synth/app.	60.0	73.9	Expert	51	49	—	53	49	Novice	53.0	39.2	Novice	44.4	46.7	—
PS, strategies	73.8	83.3	—	62	58	—	67	60	Novice	60.3	36.7	Novice	46.9	51.0	Expert
PS, effort	66.2	82.3	Expert	57	54	Novice	71	60	Novice	56.9	42.6	Novice	42.7	44.5	—
CC, Memoriz.	76.8	85.7	Expert	67	62	Novice	69	61	Novice	65.5	25.8	Novice	54.2	56.0	—

Comparison of CCNY first-year CREATE course outcomes with findings from previous studies using CLASS-Bio in introductory biology courses. CLASS-Bio tracks an Overall score and 7 category scores. Significant pre-/post- shifts are characterized as novice-like (Novice) or expert-like (Expert) (1). First-year CCNY CREATE classes (3 classes, n = 21 matched-pairs) made significant shifts in the expert-like direction Overall and in 5 of 7 categories. See Appendix 2, Table S1a for p values and effect sizes as well as data from full first-year CCNY cohort. Previous findings from research studies in introductory biology courses on other campuses (1, 11, 12) reported novice-like shifts Overall and in multiple categories (a. b. c); an introductory course for non-majors (d) made expert-like gains in two categories.

^a(1) Data from (1); N = 370, pre/post difference > 2 SE

^b(1) Data from (1); N = 170, pre/post difference > 2 SE

^c(11) CREATE COURSE; data from (11); N = 94, matched-pair 2-tailed t test; p < 0.05.

^d(12) CREATE COURSE; data from (12); introductory course designed for nonmajors, N = 77, matched-pair 2-tailed t test; p < 0.05. Significant expert-like shifts were seen in 2 categories (13).

CCNY = City College of New York; — = lack of significant pre/post change; conn. = connection; PS = problem solving; synth/app = synthesis and application; CC, Memoriz. = conceptual connections/memorization; signif = significant.

The upper-level students (Biology 35500; N = 56 matched-pairs) made statistically significant shifts in the expert direction in Overall score and in four categories: PS, synthesis and application; Real world connection; Enjoyment; and Conceptual connections, memorization (p < 0.05; 2-tailed matched-pair t tests; Fig. 1B, Table 1B). All-participant (N = 82 pre-, 71 post-) data documented significant shifts Overall and in PS, synthesis and application; Real world connection; and Enjoyment. Effect sizes ranged from 0.27 to 0.42 (matched-pairs) and 0.34 to 0.37 (all-participants; Table S1b, Appendix 2). Thus, students in both CCNY CREATE courses shifted significantly toward more expert-like perceptions in multiple CLASS-Bio categories while making no significant shifts in the direction of novice-like perceptions.

TABLE 1B.

CLASS-Bio outcomes in upper-level CREATE, in electives on other campuses and in a longitudinal study.

CLASS-Bio category	Upper-level CREATE			Upper-level elective Ba			Upper-level elective Db			Longitudinalc year 1—year 4
	% agree with experts		Nature of signif shift	% agree with experts		Nature of signif shift	% agree with experts		Nature of signif shift	% agree with experts		Nature of signif shift
	pre	post		pre	post		pre	post		pre	post
Overall	71.2	78.0	Expert	65	66	—	74	74	—	64.5	72	Expert
Real world conn.	80.6	87.8	Expert	72	74	—	84	85	—	78	85	Expert
Enjoyment	79.0	87.8	Expert	68	74	Expert	85	85	—	74	82	Expert
PS, reasoning	82.1	86.4	—	75	72	—	83	83	—	74	80	Expert
PS, synth/app.	52.8	64.6	Expert	60	65	Expert	58	59	—	54*	60*	—
PS, strategies	75.7	76.2	—	68	70	—	75	74	—	65*	70*	—
PS, effort	74.8	77.2	—	67	69	—	77	76	—	67*	70*	—
CC, Memoriz.	73.3	81.4	Expert	67	67	—	81	81	—	66	74	Expert

Comparison of CLASS-Bio outcomes in CCNY upper-level CREATE courses with findings from previous studies in upper-level electives and across a 4-year college experience. Data from ((1); a, b) and a study comparing first-year with 4^th year students ((13; c). Upper-level CREATE classes (4 cohorts, n = 57 matched-pairs) made significant shifts in the expert-like direction Overall and in the majority of additional categories. See Table S1b, Appendix 2 for p values and effect sizes for these matched-pairs, and data from full cohort (all-participants). Upper-level electives in studies by others (a, b) showed no significant shifts Overall and expert-like shifts in few (a) or no (b) additional categories. Students compared in year 1 and year 4 of their college careers (c) made shifts comparable with those of the CREATE course.

^aData from (1), n = 126, pre/post difference > 2 SE.

^bData from (1), n = 81, pre/post difference > 2 SE.

^cData from a study comparing first-year with graduating students (13), n = 83 matched-pairs, 2-tailed t test,

^*values estimated from (13), Fig. 1.

CCNY = City College of New York; — = lack of significant pre/post change; conn. = connection; PS = problem solving; synth/app = synthesis and application; CC, Memoriz. = conceptual connections/memorization; signif = significant.

Critical thinking shifts in CCNY CREATE courses

We reported previously that first-year CREATE students made significant gains on the Critical-thinking Assessment Test (CAT) across a semester (10). In the present study, upper-level CREATE students made parallel and statistically significant gains, (p < 0.05; paired t test; Table 2), on the CAT. This finding suggests that the shifts in perception tracked by CLASS-Bio (Fig. 1, 2) occurred concurrently with gains in critical thinking skills.

TABLE 2.

Outcomes on critical-thinking assessment test; first-year and upper-level CREATE students.

Course	Pre- Ave (SD)	Post- Ave (SD)	N	Significance	Effect size
Introduction to scientific thinkinga (first-year CREATE)	9.6 (2.5)	13.0 (4.4)	15	p < 0.05	0.97
Analysis of scientific literature (upper-level CREATE)	14.7 (4.1)	16.35 (3.7)	20	p < 0.05	0.42

Gains made on CAT post-course (Post) vs. pre-course (Pre). The CAT was administered to the fall 2011 Biology 10050 class* and the fall 2013 Biology 35500 class at CCNY and sent to the CAT team at Tennessee Technological University for scoring. No students from the 2011 course were in the 2013 course. Significance: Matched-pair 2-tailed t test. Effect size: Cohen’s d (16). Scoring and data analysis provided by Tennessee Technological University CAT team (15).

^aData from (10).

CREATE = Consider, Read, Elucidate hypotheses, Analyze and interpret the data, Think of the next Experiment; Ave = average; CAT = Critical-thinking Assessment Test.

We calculated normalized learning gains (LGs; 17) for the CREATE classes’ average pre-to-post-course change on each CAT question. First-year and upper-level classes showed substantial class-average normalized LGs on the following three CAT prompts: “Summarize the pattern of results in a graph without making inappropriate inferences” (Evaluation and interpretation (E/I) subcategory; first-year LG 43%; upper-level LG 100%); “Determine whether an invited inference is supported by specific information” (E/I; first-year LG 62%, upper-level LG 29%); and “Use basic mathematical skills to help solve a real-world problem” (Problem solving subcategory; first-year LG 35%, upper-level LG 50%). The first-year CREATE class also made substantial shifts on “Evaluate whether spurious information strongly supports a hypothesis” (E/I; 57% LG), and the upper-level class on “Evaluate how strongly correlational-type data supports a hypothesis” (Effective communication subcategory; 34% LG). We note that E/I, the area of greatest learning gain, may reflect CREATE courses’ emphasis on understanding figures, analyzing and discussing data, and defining the hypotheses likely to have driven the experiments that were analyzed (7).

Limitations of the study

The students in our study were also taking other courses during their CREATE semester, such as fundamental mathematics (first-years) and other STEM electives (upper-level). We thus cannot be positive that the documented CLASS-Bio shifts and CAT gains are due to the CREATE courses specifically. As we did not administer the CAT test to each of the seven sections discussed in this paper, due to financial and time limitations, we do not know whether every cohort made critical thinking gains. However, these outcomes, affective gains on the CLASS-Bio and cognitive gains per the CAT, align with previous findings of both affective (including epistemological) and cognitive (including critical thinking gains documented using CAT or a different instrument) gains in CREATE courses, both at CCNY (6, 9, 10) and at diverse four-year (18) and two-year (19) campuses, the latter in courses taught by alumni of CREATE Faculty Development Workshops (20).

Some areas of student thinking, for example epistemological views of science, are quite resistant to change and unlikely to shift due simply to ‘maturational factors’ associated with taking any semester-long course (see Hoskins et al. (9) for a discussion of this point and Jensen et al. (21) regarding challenges of linking observed gains to their causes). Other aspects of epistemological maturation across a semester of instruction have been documented previously in both first-year (10) and upper-level (9) CREATE courses at CCNY, using the Survey of Student Attitudes, Abilities and Beliefs (SAAB; 9). Neither epistemological beliefs nor gains in students’ self-rated attitudes and beliefs changed significantly in pre-/post- SAAB scores of a comparison (non-CREATE) biology course in previous work at CCNY (10), arguing that changes measured by the SAAB in CREATE courses are not likely to be an ‘automatic’ effect of taking any biology course. Similarly, we think it likely that the changes observed in CREATE classes with CLASS-Bio likely relate to CREATE teaching/learning strategies. The single-term studies by others that we use here to compare outcomes (1, 12, 13) did not include control or comparison courses and found few or no expert-like gains on CLASS-Bio (see below). If development of expert-like perceptions were a natural outcome of taking a college science course, it would be reasonable to expect substantially more gains in the direction of expert-like perceptions to have been documented.

DISCUSSION

Our results support the hypothesis that, across one semester, the perceptions of students in CREATE courses measured by the CLASS-Bio instrument would move significantly in the expert-like direction. The shifts observed indicate a maturation of understanding of the nature of biological knowledge and increased scientific thinking expertise (1). Significant shifts were seen Overall, and in categories related to problem solving, decreased use of memorization, increased enjoyment of science and enhanced understanding of conceptual connections. On a separate assessment, students in both CREATE courses also made significant pre-/post- gains on the Critical thinking Assessment test (first-year effect size (ES) = 0.9; upper-level ES = 0.4). The CAT gains were associated with substantial learning gains regarding ability to evaluate/interpret data and graphs. The findings suggest that CREATE courses are likely to have strong impacts on students’ perceptions of the nature of scientific knowledge and scientific thinking ability, beyond simple gains in content or conceptual understanding.

Previous use of CLASS-Bio to examine student vs. expert perceptions

CLASS-Bio has not yet been used extensively in education studies, and results to date are mixed. Across six large-enrollment Introductory Biology courses on two campuses, five cohorts’ Overall scores were significantly lower post-course, indicating that students’ perceptions became more novice-like during a term of instruction (1). All significant shifts in individual categories were also novice-like (Table 1A). Similar shifts toward novice-like perceptions in large introductory courses have been reported in physics (22, 23) and chemistry (24), using CLASS surveys for these disciplines. It is not possible to attribute the shifts toward more naïve perceptions specifically to the introductory courses, as the students in the studies were also taking other classes. It is, however, notable that in three STEM disciplines, shifts toward significantly more naïve perceptions across a semester of introductory instruction have been documented.

A recent study used CLASS-Bio to compare a large introductory biology course for nonmajors with a separate course for majors on the same campus (11, 12). The two courses studied were designed with different goals, though both “covered” introductory biology. Majors showed significantly more novice-like perceptions post-course in every category of CLASS-Bio. Nonmajors shifted in the novice-like direction in one category and in the expert-like direction in two (Table 1A). While the courses served different student populations (and thus, there were differences beyond curriculum), the outcome may point to curricular design as a factor shaping students’ perceptions about science.

Previous studies using CLASS-Bio with upper-level students have documented some maturation of students’ perceptions. For example, in upper-level biology electives on several campuses, Overall scores did not change significantly across a term of instruction, but post-course scores in some categories reflected significant shifts toward more expert-like perceptions (1). In contrast to outcomes in first-year introductory biology courses, it is notable that neither upper-level elective cohort made novice-like shifts (Table 1B). In a longitudinal study of biology students’ thinking over a four-year college “career,” CLASS-Bio surveys were given in year 1 and again in year 4 (Table 1B). Expert-like shifts were seen Overall, and in four categories (13). While the outcomes must be interpreted with caution as far fewer students completed the “Year 4 post-” than the “Year 1 pre-” survey, the data suggest that at some point during their college years, students’ perceptions in some areas matured. As the study did not sample intermediate time points, it is not known when in their college careers these students’ perceptions became more expert-like.

In sum, previous studies using CLASS-Bio in semester-long courses have not documented substantial maturation in students’ perceptions along the novice-expert continuum. In large introductory biology courses, students’ perceptions often underwent novice-like regression rather than expert-like advancement, mirroring previous findings in introductory physics and chemistry courses. In upper-level electives, students’ views for the most part did not change significantly across a term of instruction.

Course characteristics as related to CCNY CLASS-Bio outcomes

Typical Introduction to Biology courses combine a three- or four-hour weekly lab with one or more lectures of 60 to 90 minutes’ duration. Upper-level biology electives commonly meet weekly for three to four hours, or two or three times per week for one and a half to two hours. Thus, CREATE first-year courses are shorter in duration than typical introductory courses, and CREATE upper-level courses are comparable in duration with other upper-level electives. One interpretation of the comparison of CREATE CLASS-Bio outcomes with others (Tables 1A, 1B) could be that large science classes do not promote shifts in students’ perceptions and scientific thinking while small ones do. We think this unlikely. For example, some significant expert-like shifts were made on individual statements that were later included in CLASS-Bio, in a large (N > 200) introductory biology course extensively redesigned to employ state-of-the-art pedagogical techniques (25). In addition, gains in some categories were seen in the large (80–126 students) biology electives that were part of the original CLASS-Bio study (1). On the campus where nonmajors made a few expert-like gains while majors made none (11, 12), the majors’ course (N = 94) concentrated on biological detail and theory; all seven CLASS categories showed significant novice-like shifts across the semester. The nonmajors’ course (N = 77) took a broader perspective, including a focus on the societal value of science and technology; two CLASS-Bio categories showed significant expert-like shifts and only one registered a novice-like shift. The findings argue that it is not simply a small-group learning environment that underlies positive CLASS-Bio shifts, and that significant shifts in the direction of expert-like perceptions and thinking can be promoted in large classes. It can be challenging to define the factors that underlie gains; an interesting recent study focused on conceptual learning (not on student novice/expert perceptions of science) suggests that outcomes attributed to a particular course design (flipping) could be primarily due to implementation of active engagement approaches (21). Further research will be needed to fully parse out the factors which underlie students’ shifts toward more expert-like perceptions, when they occur.

Interestingly, physics students in a large “reform-oriented course” (N = 397) did not shift toward more expert-like perceptions on CLASS-Physics (22). However, in a course reconfigured to focus on “Physics and Everyday Thinking,” significant shifts were seen, even in classes with up to 100 students (26). We think it likely that CREATE’s emphasis on scientific thinking, not class size, is key to students’ shifts on CLASS-Bio. In this regard, it is interesting that a recent study on changing a course’s focus to modeling and away from more traditional curricula led to large expert-like shifts on CLASS-Physics (27).

As students in Introductory Biology courses with different emphases score differently on CLASS-Bio (11, 12), course design likely influences students’ perceptions. Many upper-level CREATE students’ previous experience with primary literature involved “presentations” in which data were recapitulated and accepted without question. First-year students had not used such literature previously. Taking time to work carefully through study design and methods, and to closely examine data encourages CREATE students to construct evidence-based critiques and draw their own conclusions. These processes emphasize analytical and creative thinking, arguably supporting deeper understanding of bona fide scientific process. Classroom discussions likely increase students’ awareness both of how scientific knowledge develops and how it changes. Encouraging students to consider such issues may in turn stimulate intellectual development beyond understanding of basic concepts/content (28–31) and contribute to the maturational shifts seen on CLASS-Bio.

Novice-like and expert-like thinking in science

In a discussion of inquiry in science, Shavelson and Towne noted that “[t]he culture of science fosters objectivity… through concerted efforts to train new scientists in certain habits of mind. By habits of mind, we mean things such as a dedication to the primacy of evidence, to minimizing and accounting for biases that might affect the research process, and to disciplined, creative, and open-minded thinking” (32, p. 53). We speculate that the CREATE strategy may support development of habits of mind more typical of experts than of novices by coupling a focus on scientific literature with diverse in-class activities that work synergistically to shape students’ thinking and deepen their understanding of the nature of science.

The nature of expert thinking in science is an issue of ongoing investigation (33). In a report on expert vs. novice thinking (not involving CLASS survey instruments), experts are described as noticing “features and meaningful patterns of information that are not noticed by novices.” Experts have also “acquired a great deal of content knowledge that is organized in ways that reflect a deep understanding of their subject matter” (34, p. 31). In CREATE classes, complementing pre-class homework with in-class discussion of scientific data may build students’ confidence and strengthen their ability to organize and apply their content knowledge. This ability in turn may improve students’ skill at recognizing patterns of information.

CONCLUSIONS

The finding that first-year and upper-level CREATE students make multiple significant shifts toward more expert-like perceptions of science across a semester, as measured by CLASS-Bio, contrasts substantially with previous findings by others using this instrument. Just as other CLASS-Bio studies could not attribute students’ primarily naïve-like shifts in perception (in introductory courses) or relative lack of shifts (in electives) directly to the classes under study, we cannot attribute our students’ gains directly to their CREATE courses. We note above, however, that our previous studies suggest that CREATE courses have effects that are not achieved maturationally. The paucity of expert-like shifts documented on CLASS-Bio in studies by others also suggests that students do not “automatically” develop more mature perceptions of science across a term of instruction. Given that CREATE courses have been successful beyond CCNY, evoking diverse cognitive and affective gains in cohorts of students at other four-year (18) and two-year (19) colleges/universities (in studies that did not include assessment with CLASS-Bio), we suggest that this scientific-literature-based strategy could be a way to encourage maturation of students’ perceptions of biology, as well as scientific thinking skills, on a broad scale. Implementing CREATE does not require special equipment or materials, so the strategy is adaptable to a wide range of colleges and universities. Adding scientific-literature-focused CREATE courses to traditional curricula has the potential to foster students’ development of a more sophisticated understanding of science and to help them refine their ability to think like science experts.