“Is This Class Hard?” Defining and Analyzing Academic Rigor from a Learner’s Perspective

Abstract

Despite its value in higher education, academic rigor is a challenging construct to define for instructor and students alike. How do students perceive academic rigor in their biology course work? Using qualitative surveys, we asked students to identify “easy” or “hard” courses and define which aspects of these learning experiences contributed to their perceptions of academic rigor. The 100-level students defined hard courses primarily in affective terms, responding to stressors such as fast pacing, high workload, unclear relevance to their life or careers, and low faculty support. In contrast, 300-level students identified cognitive complexity as a contributor to course rigor, but course design elements—alignment between instruction and assessments, faculty support, active pedagogy—contributed to the ease of the learning process. Overwhelmingly, all students identified high faculty support, learner-centered course design, adequate prior knowledge, and active, well-scaffolded pedagogy as significant contributors to a course feeling easy. Active-learning courses in this study were identified as both easy and hard for the very reasons they are effective: they simultaneously challenge and support student learning. Implications for the design and instruction of rigorous active-learning college biology experiences are discussed.

INTRODUCTION

A hallmark of a high-quality undergraduate education is academic rigor (Graham and Essex, 2001). Although accreditation standards provide a benchmark for academic rigor across institutions (Wergin, 2005), few discipline-specific accreditation bodies exist for biology (e.g., Cheesman et al., 2007; American Society for Biochemistry and Molecular Biology, 2013), compared with other disciplines like chemistry and engineering (American Chemical Society, 2015; Accreditation Board for Engineering and Technology, Inc. [ABET], 2017). Therefore, within and between institutions, departments have a high degree of intellectual freedom to design and assess their courses, which may result in varying degrees of academic rigor (Dill et al., 1996). Given the range of experiences students can have in different programs under these conditions, there is a growing need to define academic rigor in the discipline of biology, especially as academic programs are responding to calls for reform (American Association for the Advancement of Science, 2011).

Defining Rigor

Academic rigor is a challenging concept to define, as there are varying perceptions of what rigor means, and correspondingly few papers to offer a clear definition of the term. For example, some define rigor as “academically demanding” (Wyatt, 2005), “fast-paced” (Winston et al., 1994), and needing a high degree of “energy and time” on behalf of the student (Winston et al., 1994). Others define rigor based on attributes of the instructor, such as possessing a terminal degree in the discipline and full-time status (Clinebell and Clinebell, 2008). Still others explicitly define rigor based on cognitive expectations—for example, the depth of questions asked of students in class and on assignments (Braxton, 1993), the connection between concepts (Nicholson [1996] in Graham and Essex [2001]), or the amount of critical thinking (Taylor and Rendon, 1991). The National Survey of Student Engagement (NSSE), a widely used college assessment, includes a subscale relevant to the level of academic challenge. The NSSE subscale includes questions on both workload/difficulty of courses (e.g., amount of reading, writing, and work required) and questions on higher-order thinking skills of analyzing, synthesizing, and applying, suggesting that both hard work and a cognitive challenge are part of the definition of academic rigor.

Taken together, these definitions of academic rigor seem to suggest that students need to learn how to think critically (Payne et al., 2005), engage with concepts that require deep thought and effort (Winston et al., 1994), and make connections between concepts. It is important to note that, in certain contexts, academic rigor is argued to be disconnected from real-world relevance (e.g., Clinebell and Clinebell, 2008), focusing more on abstract or esoteric lines of thinking accessible only to select niche audiences. Alternatively, others argue that rigorous learning must be meaningful and relevant, focused on connections to practical problems (Draeger et al., 2013). Given the range of perspectives on academic rigor, a guiding definition for modern biology students and instructors is timely and appropriate.

For the purposes of our work, we adopt the following definition of academic rigor: “learning meaningful content, with higher-order thinking, at the appropriate level of expectation in a given context” (Draeger et al., 2013, p. 268), leading to ownership of one’s learning (Bain, 2004). This definition can be broken down into four components: 1) learners engage in higher-level cognitive processes (Payne et al., 2005); 2) learners transfer concepts and content from scale or subdiscipline or between problems (Prosser and Trigwell, 1999); 3) learners engage in meaningful content (Jensen, 2005; Draeger et al., 2013); and 4) learners have appropriate levels of challenge and support (i.e., attainable expectations; Sanford, 1962; Graham and Essex, 2001). This study explores attributes that contribute to a rigorous course from the student’s perspective.

Student Perception of Academic Rigor

Draeger et al. (2015) found that students have a challenging time articulating definitions of academic rigor, and concluded that the term “rigor” itself may contribute to such confusion. These students tended to conceptualize academic rigor based on their perception of course difficulty, and whether a class seemed “hard.” In another study, students used words like “challenge,” “hard,” and “difficult” to explain their conceptions of academic rigor (Gordon and Palmon, 2010). These descriptors and constructs of rigor from the lens of the student were used in the design of survey items for our study.

In addition, student perception of rigor appears to be unrelated to student learning gains or end-semester course evaluations (Cohen, 1981; Uttl et al., 2017). In certain contexts, a course viewed as difficult may receive lower course evaluations, because students put forth more effort and thus perceived the course to be “harder” (Weinberg et al., 2007). This suggests that end-semester evaluations do not adequately measure objective measures of academic rigor as defined earlier. For example, they do not directly address the number and types of higher-level thinking students engage in, nor do they reflect the transfer of concepts across scales or disciplines. Even within the context of a given course, prior research suggests that most biology courses use summative assessments that measure low-level thinking by Bloom’s taxonomy (Momsen et al., 2010). Even more objective measures of assessment difficulty—for example, the degree to which an assessment question differentiates students on the basis of performance score—poses challenges for defining rigor in biology. Such psychometrically defined difficulty is not always related to cognitive challenge as measured by Bloom’s taxonomy (Lemons and Lemons, 2013; S. A. Wyse and A. E. Wyse, unpublished data). That is, it is possible to ask an easy analysis question and a challenging recall question. Therefore, based on existing assessments (course evaluation and summative assessment within a course), we know relatively little about student perceptions of academic rigor in biology and which attributes of a course influence the student viewpoint for why a course seemed “hard” or “easy.”

Interestingly, faculty notions of academic rigor guide the process of setting standards for student learning. Although it is widely accepted that higher-order cognitive skills are valuable standards that define true student achievement (American Association for the Advancement of Science, 2011), elements faculty identified as important for achieving these higher-order cognitive tasks include: prior knowledge (schema), effort, perceived difficulty, and time required on task (Lemons and Lemons, 2013). In this context, faculty perception of “rigor” is often experienced as the line between challenge and frustration, similar to the student view of easy versus hard. These factors do not, however, take into account instruction and its impact on the learner, that is, how elements such as scaffolding, alignment, learning context, and supportive environments can greatly modulate the learner frustration. Thus, rigor is an experientially defined construct, distinct from the psychometrically defined standard setting (Cizek and Bunch, 2007) common in K–12. Because it is elusively defined for faculty and students alike, yet a strong influencer of instructional choices, we aim to make sense of the experience of rigor though the student point of view.

In this study, we explore academic rigor from the experiences and perspective of the biology learner. We use the terms “rigor,” “challenge,” “difficult,” and “hard” interchangeably to reflect the student use of these terms (Draeger et al., 2015). For both introductory and 300-level biology students, we uncover what the learner means when he or she says “this course is easy and/or hard.” We sampled students enrolled in a course being taught with a “scientific teaching” approach that embraces “the same rigor as science at its best” by teaching science the way it is practiced (Handelsman et al., 2004). Prior research has established the many benefits of using these active, learner-centered approaches across science, technology, engineering, and mathematics (STEM) disciplines (Freeman et al., 2014). Yet, these practices are sometimes perceived as having questionable academic rigor (Parry, 2012), primarily because these courses allow for more processing time and constructivism reminiscent of early learning (e.g., lower elementary) compared with the traditional, lecture-based delivery style of higher education. Thus, by selecting courses taught under the umbrella of scientific teaching philosophy, we sought to gain a better understanding of academic rigor in a context in which it is not well understood relative to traditional STEM courses.

Specifically, we ask three research questions: 1) How do students define easy and hard courses? 2) How does active-learning influence what students perceive as academic rigor/challenge/difficulty? 3) How do students relate academic rigor/challenge/difficulty to learning?

METHODS

Student Population

To assess biology students’ perception of course rigor, we began by administering an end-of-the-semester survey to students enrolled in introductory biology courses at a small liberal arts institution. The students enrolled in these introductory biology courses (n = 155) represented students from biology and biology-related majors (73 and 63% for Intro Bio 1 and 2, respectively) and included mostly first- and second-year students (Table 1).

TABLE 1.

Descriptive statistics for introductory biology students^a

Course	Freshmen	Sophomores	Juniors	Seniors	Incoming GPA	ACT score	% Male
Introductory Biology 1	6	27	24	9	2.93	26.5	44
Introductory Biology 2	37	50	37	26	3.40	26.2	40

^aData include number of students at each class standing level (based on credit), incoming grade point average (GPA) and ACT scores, and the percent of the class identifying as male.

Upper-level students also completed the survey (n = 39) during the Spring of 2013. These courses contained seniors (Table 2).

TABLE 2.

Descriptive statistics for upper-level biology students^a

Course	Seniors	Incoming GPA	ACT score	% Male
Upper Level 1	19	3.54	28.68	63
Upper Level 2	19	3.57	26.35	26

^aData include number of students at each class standing level (based on credit), incoming GPA and ACT scores, and the percent of the class identifying as male.

This study was conducted under the guidelines and approval of Bethel University’s Institutional Review Board.

Course Design

All courses (both introductory and upper level) were taught using active-learning approaches under the scientific teaching paradigm (Handelsman et al., 2004). Typically class sessions were offered on a M-W-F schedule for a 70-minute duration. Lab sessions were offered on a T-Th schedule for ∼150 (introductory) or 180 (upper level) minutes. Class and laboratory used a flipped or upside-down pedagogical structure. Students completed preclass or lab reading and online activities targeting low-level Bloom’s (Anderson and Krathwohl, 2001) in advance of each face-to-face session, receiving teaching assistant feedback. In session, class time was devoted to a variety of collaborative and cooperative learning activities, including low-stakes quizzes, minilectures, jigsaw activities, audience response (“clicker”) questions, case studies, simulations, student construction of scientific models, problem solving on whiteboards, concept mapping, short data investigations, and other activities. Instructors were available in session to provide real-time feedback on these higher-order cognitive activities (Bloom’s 4–6) and experimental work (Bloom’s 3–6; Figure 1). The same two instructors (S.A.W. and P.A.G.S.) taught the 100- and 300-level courses through the duration of the study, enabling comparison among sections.

FIGURE 1.

Representative example of lesson design elements that promote academic rigor. (A) Scaffolding learning progression (criteria 4). Appropriate levels of challenge and support are achieved through a flipped-classroom structure, in which students engage in low-level Bloom’s pre-activities activities, then spend class time extending learning through higher-order group activities and models. TA, teaching assistant. (B) Representative example of summative assessment item meeting criteria 1–3. Learners engage in higher-order processing spanning biological scales using a cross-cutting concept of evolution through a meaningful case study.

Survey Development

The survey was designed to get students to articulate their definitions of academic rigor by triangulating data from several questions. Questions were developed based on the Michael (2007) survey asking students to identify what makes physiology easy/hard to learn. An additional open-ended question was derived from interview prompts asking students to describe a learning experience they had in college and what made that experience so rigorous (Draeger et al., 2015). The survey was peer reviewed by three colleagues, and then pilot tested with students (n = 44). Following pilot testing, we made necessary wording changes to improve the quality of the data we obtained. The final open-ended survey included four questions:

1. Please describe the hardest class you have ever taken (at college). What made it so rigorous?

2. Please describe the easiest class you have ever taken (at college). What made it so easy?

3. Please describe whether this class (course name) was easy or difficult for you. What made it that way?

4. Please describe a course that you’ve taken where you felt you learned the most. Why did you learn so much?

These revised and field-tested surveys were then administered to 194 students enrolled in an active-learning biology course in the Fall of 2012, Spring of 2013, and Fall of 2013 at either the 100 level (n = 155) or the 300 level (n = 39).

Coding

Following administration, we used qualitative coding methods (Bogdan and Biklen, 1998) to discern patterns in the data set. A coding rubric for each question was developed (Tables 3–7). Three researchers worked together to develop the coding categories and determine decision rules for binning particular response comments into each category. We constructed a single coding rubric for questions 1 (Table 3), 2 (Table 4), and 4 (Table 6). For question 3, we generated three rubrics to account for the variety of responses (Table 5A, this course was easy; Table 5B, this course was hard; Table 5C, this course was a combination of easy and hard). Beginning with coding a small subset of the data, the three raters coded each transcription. Areas of disagreement were discussed until consensus was reached and decision rules were refined. This process continued by small-batch coding the entire data set and then revisiting the data set for consistency in coding.

TABLE 3.

Coding rubric for survey question 1: Please describe the hardest class you have ever taken (at college). What made it so rigorous?

Category	Definition and examples
Low preparation and interest	Students do not have an interest in the concept/course or do not have adequate preparation for the course.
High workload	Course requires a lot of time, especially out of class (especially with other people), assignments, reading, and information.
Quick pace	Course has too much content covered too quickly for a student to process.
Unclear importance	Students struggle with determining what is important; there are lots of facts to be memorized; facts don’t seem to “fit” anywhere to students. Disorganized course structure makes it hard for students to follow.
Lack of alignment	Assessments do not match content or approach in the class. Most of the grade depends on one or two ­assessments.
Low faculty support	Faculty do not appear to help and support students; lots of learning being done on their own (independent learning); lack of active-learning approaches; students receive little feedback on assessments for ­improvement.
High cognitive demanda	Material/content is more complex and requires critical thinking, application, analysis, synthesis, and/or evaluation.

^aIf students specifically mentioned a hard assignment and referenced cognitive demand (e.g., critical thinking), then we coded the response as “HCD” (high cognitive demand); if not, then we did not code that mentioning, as we could not tell what was “difficult” or “hard” about the assignment or what “hard” meant in this context.

TABLE 4.

Coding rubric for survey question 2: Please describe the easiest class you have ever taken at college. What made it so easy?

Category	Definition
Strong preparation and interest	Student has strengths in this content area, is interested in the course, and/or has prior course work/learning in this area that leaves the student feeling prepared for the course.
Low/manageable workload	Easy courses have low student workload expectations, or there are “reasonable” workload expectations (e.g., “not too many big projects”).
Course content is “logical”	Course material is “commonsense” or course content seems “logical” to students.
Clear alignment	Expectations for what is important are clear; class content and exams are well matched.
High support	Faculty help, listen, and provide support (e.g., peers) for studying; active-learning strategies are employed; feedback is provided.
Low cognitive demand	Material/content is fact based, requiring memorization or simple comprehension. Critical thinking and higher levels of Bloom’s taxonomy are absent.

TABLE 5.

Coding rubric for survey question 3: Please describe whether this course (name) was easy or difficult for you. What made it that way?

	Definition
A. Category: Easy
High preparation and interest	Student has strengths in this content area, is interested in the course, and/or has prior course work/learning in this area that leaves the student feeling prepared for the course.
Low/manageable workload	Easy courses have low student workload expectations, or there are “reasonable” workload expectations (e.g., “not too many big projects”).
Course content is “logical”	Course material is “commonsense” or course content seems “logical” to students.
Clear alignment	Expectations for what is important are clear; class content and exams are well matched.
High support	Faculty help, listen, and provide support (e.g., peers) for studying.
Cognitive demand	Material/content is fact based, requiring memorization or simple comprehension. Critical thinking and higher levels of Bloom’s taxonomy are absent.
B. Category: Hard
Low preparation and interest	Student does not have an interest in the concept/course or does not have adequate preparation for the course.
High workload	Course requires a lot of time, especially out of class (especially with other people), assignments, and reading.
Quick pace	Course has too much content covered too quickly for a student to process.
Unclear importance	Students struggle to determining what is important; there are lots of facts to be memorized and facts don’t seem to “fit” anywhere to students; disorganized course structure makes it hard for students to follow.
Lack of alignment	Assessments do not match with content or approach in the class; most of the grade depends on one or two assessments; students receive little feedback on assessments for improvement.
Low faculty support	Faculty do not appear to help and support students; lots of learning being done on their own (independent learning).
High cognitive demand	Material/content is more complex; requires critical thinking, application, analysis, synthesis, and/or evaluation.
C. Category: Both
Lack of preparation made it hard.	Prior knowledge in the content area, skill or needing to transfer information from other content areas
High cognitive load made it hard.	Material/content is more complex; requires critical thinking, application, analysis, synthesis, and/or evaluation. There is more math.
The approach made it easy.	Course incorporated peer groups, faculty support, structure/scaffolding, and/or review.

TABLE 7.

Forced-choice survey questions

Topic	Question stem	Question choices
Hard	Think about the hardest class you have ever taken in college. Which of the following contributed to it being so difficult? Select as many as apply.	My lack of prior preparation made the course difficult. I wasn’t interested in or motivated for the course. The course had too much work. The pace was too quick for me to keep up. It was hard for me to determine what was important. Tests/exams did not align with what was taught in the course. I had to do a lot of learning on my own. The instructor provided very little feedback on my course progress. The course material was really complex (e.g., required a lot of critical thinking). Other
Easy	Think about the easiest course you have ever taken in college. Which of the following contributed to it being so easy? Select as many responses as apply.	I was really interested in the course material. I had a strong background and was prepared for the course. The workload was manageable. The content was logical; it made sense. The way the content was presented made it easy to follow. The expectations were clear. The exams and other assessments matched what we learned and did in class. The instructor provided feedback, help, and support. The content was simple and did not require a lot of critical thinking. Other

TABLE 6.

Coding rubric for survey question: Describe a course that you’ve taken where you felt you learned the most. Why did you learn so much?

Category	Definition
Interest and utility	Students are personally interested in the content area or know it will be useful to them in the future; material is related to past or future learning.
Workload	Workload is high.
Faculty support	Faculty support student learning; instructor is “excellent”; students offered “this faculty was…” statements.
Application	Students can see how material applied or mattered to “real life.”
Convergence of past learning experiences	Students can see disciplines coming together; e.g., combining “math & chem with Bio”; faith integration (students see concepts relating to and/or enhancing their personal faith); real-life application (content connects to a real-world case or context, job/career they are interested in); personhood (helps them develop their academic identity, e.g., science identity).
Volume	Amount of content, greater detail
High faculty expectation	Students perceived that faculty had high expectations.

Creation of the Forced-Choice Survey

Coding category patterns were then used to generate forced-choice responses for subsequent survey administrations. The question prompt read: “Think about the hardest (easiest) class you have ever taken in college. Which of the following contributed to making it so difficult (easy)? Select as many as apply.” The selection options were derived from coding categories (see preceding section; Tables 3–6). Students were also given an “other” option to include anything they did not find on the list (Table 7).

We administered the forced-choice survey questions at the end of the semesters of Spring 2014 and Fall 2014 to increase the sample size (n = 90 additional 100-level students, n = 20 additional 300-level students). Student demographics and course composition were similar in these 100-level courses to those listed in Table 1. In total, 304 students (n = 245 for 100-level students, 59 for 300-level students) contributed responses to the data set between Fall 2012 and Fall 2014.

Analysis

To determine response pattern differences between 100- and 300-level students, we calculated frequencies for each response code and compared them. A chi-square test was used to differentiate the overall patterns in response distributions between these students. In addition, quantitative responses from forced choices were averaged with standard errors computed, and chi-square tests were also used to compare patterns in frequency distributions between 100- and 300-level students.

RESULTS

How Do Students Define Easy and Hard Courses?

At both the 100 and 300 levels, students universally characterized hard classes as having a high workload. In addition, poor background preparation and/or low interest in the subject matter caused both groups of students to perceive a course as harder:

“Professor expected a ridiculous amount of reading that nobody had time for and probably wouldn’t take much from it anyway. Then took random little facts from book that we never went over in class and put them on test. Also no review and did not put much stuff we learned in class on the tests.”

However, 100-level student perception of course difficulty was especially coupled to low faculty support, as well as a fast pace, and facts that have unclear relevance or importance:

“The hardest class I ever had taken in college focused a lot of learning on the go and by myself. The teacher was almost too smart for the class and he was not around much after class to help with questions. The assignments and projects were very difficult and a lot of the learning was expected to be done on your own.”

These factors contributed less to course difficulty in the view of their 300-level counterparts. Instead, 300-level students particularly noted that high cognitive demand contributes to course difficulty, as does high workload and time demands, poor alignment between instructional practices and assessment items:

“[Biology course] had lots and lots of information to take in, and the tests were fairly difficult—asking the students to apply what they learned to bigger models/life scale. The labs were also time-consuming.”

Overall, the factors identified by 100- and 300-level students about what constitutes a difficult course differed (χ² = 39.016, df = 6, p = 0.00000071; Figure 2).

FIGURE 2.

Student perceptions of what makes a college biology course hard. The 100-level students (black, n = 155) defined hard courses as those that have a high workload and low faculty support and seem to focus on disparate facts the students struggle to see as important. The 300-level students (gray, n = 39) recognized workload and preparation as key factors that make courses hard, but also noted that higher cognitive levels make courses harder. Overall response patterns differ for 100- and 300-level students (χ² = 39.016, df = 6, p = 0.00000071).

Students defined courses as easy when they had the appropriate background to be successful in the course, a manageable workload, and high faculty support. Faculty supports include a teaching style that engages the learner, course design and setup, the use of collaborative learning, reoccurring review of concepts, visual and model-based learning, as well as affective characteristics such as being encouraging, believing in students as learners, and contributing to a nonstressful learning environment. Additionally, course content following logically from one key idea to another contributed to the perception that a course was easy. It also helped learners when instructional objectives aligned well with assessments. Students also identified low cognitive demand as a factor for making a course seem easy:

“The easiest class I have taken in college had a lot to do with the content that I already knew…. The information content was very common sense like and there wasn’t much at all. Class that require no critical thinking are easy.”

Students commented on their own role in learning by recognizing that being highly prepared for the course and interested in the material makes the course seems easier compared with one in which they were underprepared or lacked interest. The factors identified by 100- and 300-level students about what constitutes an easy course were similar (χ² = 2.46, df = 5, p = 0.7825; Figure 3).

FIGURE 3.

Student perceptions of what makes a college biology course easy. The 100-level students (black, n = 155) defined easy courses as those that have a low or manageable workload, logical content, personal interest, and/or strong background preparation by the student, and high faculty support. The 300-level students (gray, n = 39) recognized easy courses as having logical content, a manageable workload, clear alignment, high faculty support, and, last, a low cognitive demand asked of them in the course.

When provided forced choices to choose from, 100-level students attributed a hard course to their lack of prior knowledge and the degree to which the course required them to apply material. The 300-level students identified a high workload as a main contributor to difficulty (Figure 4). Both 100- and 300-level students identified cognitive demand (e.g., application and case study) as contributing to making a course rigorous. These data are consistent with the open responses in Figure 2, but notably, faculty support was not identified in the forced-response survey as it was in the retrospective analysis (Figure 2). With respect to factors that contribute to a course feeling easy, both 100- and 300-level students identified that the way the course was taught made the course seem easy (“I didn’t feel like I was learning”). This perception, coupled to other factors students identified as high “faculty support,” suggests that students recognize the importance of course delivery and design and teaching approaches as factors contributing to the seamlessness and ease of a course (Figure 4). These factors provide appropriate challenge and support so that the students moved from their current knowledge to newly acquired knowledge in a manner that was not stressful (“I didn’t feel like I was learning”). Patterns for hard (χ², df = 2, p = 0.036) and easy (χ², df = 3, p = 0.025) attributes of courses differ between 100- and 300-level students (Figure 4).

FIGURE 4.

Student self-reports from selected “choose from” response items of what makes a course hard or easy. The left side of the graph corresponds to hard attributes, and the right side of the graph corresponds to easy attributes. 100-level students (black, n = 90) identify lack of preparation and workload as reasons courses were hard. Both 100- and 300-level students acknowledge the required application of course content contributed to its difficulty. 300-level students (gray, n = 40) attributed courses as easy due to their background and the “ease“ of learning (I didn’t feel like I was learning).

How Does Active Learning Influence What Students Perceive as Academic Rigor?

To unpack student perception of what makes an active-learning classroom seem hard or easy, we asked students enrolled in an active-learning classroom modeled after a scientific teaching philosophy (Handelsman et al., 2004) whether “this course” was easy or hard, and why. Just shy of half of the students (47%) recognized their active-learning course to be both easy and hard, while 29% found the course hard and 24% found the course easy. Students overwhelmingly identified that active-learning courses were easy because of the high level of faculty support provided to learners at both the 100 and 300 levels (Figure 5). Similar to the definitions identified earlier (Figure 3), students characterized that ease was based on having high faculty support, proper background preparation and high interest in the content, clear alignment between learning objectives and assessments (i.e., course content and exams), logical course content, and manageable workload.

“It was rigorous with the amount of information but easy with the way the class was set up. Groups were great, objectives helped and it was a lot of repetition.”

“Some of the material was hard but it was taught in a way to help us understand how it connected as a whole.”

FIGURE 5.

Student perception of what makes “this” active-learning college biology course easy. The 100-level students (black bars, n = 41) defined the active-learning course as easy. The lowest contributor was the cognitive demand. The 300-level students (gray bars, n = 11) recognized the active-learning course as easy, indicating that it was easy because the content was logical, there was a high degree of learning support provided by faculty members, and it had a manageable workload and clear alignment.

Patterns did differ between 100- and 300-level students (χ² = 25.28, df = 5, p = 0.000127) due to a few more 300-level students identifying logical content and clear alignment as course attributes that contribute to a course being easy. Notably, a very small percentage of 100-level students and no 300-level students said these courses were easy because there was not enough cognitive challenge.

For the students who identified “this active-learning course” as hard (n = 45 for 100-level students, n = 5 for 300-level students), reasons were largely focused on the level of cognitive challenge (i.e., cognitive demand, high Bloom’s) required of them in the course (Figure 6).

“[This course] was a more rigorous class. It required a lot of work and a lot of thinking (rather than just memorization). Labs also made us think outside of the box. Our tests did this as well. It was much more beneficial to be able to apply the things we learned to actual life situations rather than multiple choice questions.”

FIGURE 6.

Student perception of what makes “this” active-learning college biology course hard. The 100-level students (black bars, n = 45) defined the active-learning course as hard and subsequently characterized that difficulty as arising from the cognitive demand required of them in the course, the workload, and their lack of preparation for the course. The 300-level students (gray bars, n = 5) recognized the “active-learning course” as hard, with all students defining it as hard because of the high cognitive demand.

Secondarily, both groups identified a high workload and low preparation as factors contributing to a hard course experience.

“This class was fairly difficult in a very, very non-stressful manner. I was not familiar with nearly everything that we learned, so there was so much to learn. It was also decently time-consuming, especially with the data set analyses. The amount of work we did with the data sets was probably the most difficult part of the class, but I gained so much knowledge from them.”

Only 100-level students identified pace and unclear importance of content as factors making the active-learning course hard (Figure 6). Low faculty support was not mentioned by either level of students, nor was lack of alignment. Patterns in responses for 100- and 300-level students differed, largely due to the absence of responses in the last four categories (χ² = 21.61, df = 6, p = 0.0014). Taken together, these data strongly suggest that the perceived ease in an active-learning course taught under the scientific teaching paradigm is due to learner-centered design and pedagogical choices consistent with the philosophy of “teaching science as science is practiced.”

This interpretation is corroborated by a subset of the data wherein students reported that the active-learning course was experienced as both easy and hard (n = 72 for 100-level students and n = 8 for 300-level student). In these comments, students were able to differentiate which aspects of the course experience gave these impressions. Those who cited specific attributes of the course as hard deemed it so as a result of their own lack of preparation or due to the high cognitive load for the course. At the same time, these students identified attributes of the course as feeling easy due to the pedagogical approaches used in the course (e.g., scaffolding, cooperative learning).

“This course was easy, not because the material was easy, but because of the way it was taught. Learning in groups and through projects made the harder material easier to learn and having clear expectations and objectives really helped. I was also never overwhelmed by the amount of work in this class which was really great because it kept me motivated to keep working and to put in my best effort on all my assignments. I think the hardest parts of the course were making models. That process, though hard, was really beneficial because we had to find the answers for ourselves instead of just being given the answer. Thinking through the model was challenging but very beneficial in the end.”

How Do Students Define Academic Rigor in Relationship to Learning?

To understand how students relate their perception of academic rigor to their learning, we asked them to describe courses in which they learned the most. Students at both the 100 and 300 levels identified that they learn the most in courses in which faculty support of their learning is high, in which they have interest in or see the usefulness and applicability of the course content, and in which they were not previously well-educated in the course material (χ² = 11.229, df = 9, p = 0.26; Figure 7). Secondarily, students associated a relatively high level of learning to courses in which volume, workload, and faculty expectation were high. Interestingly, these were previously identified as factors making a course hard (Figure 2). Application of content also influenced student perception of courses in which they learned the most (Figure 7). Other factors, such as synthesis of other learning experiences, character building, and the opportunity for metacognition and reflection were also cited by students.

“I put a lot of work into the class. It was a very small group and a lot was expected from each student. The teacher was very passionate about the subject and showed a great desire for us to truly learn.”

“I feel as if I learned the most in [course]. I believe that I learned so much because we were expected to know and understand the material for tests, as well as create models to communicate concepts in class. The classes were also set up to facilitate learning, we had a overview of the learning objectives at the beginning of the class, and a review at the end of class. We were also asked to brainstorm our own ideas, and come up with applications of the material in our everyday lives.”

“I’ve never been good at connecting concepts to each other and to real life situations until [course]. I contribute [sic] my learning to the stress level of the course, the clicker questions, sitting in a group, four minute summaries and learning objectives. Not having to spend hours over a textbook trying to grasp what was taught in class helped me relax and stay happy. I was not overwhelmed with the workload of this class which allowed for me to take my time and have learning objectives sink in, instead of rushing to do homework every day while not having time to understand why I got the answer.”

FIGURE 7.

Comparison of 100- and 300-level student responses to courses wherein they learned the most. Bars represent frequencies of response. Both 100- and 300-level students identified faculty support as being key in a course in which they learned the most. In addition, courses in which they did not have a lot of initial knowledge but had high interest and saw strong practical application resulted in their perception of learning the most.

Taken together, these data suggest that students learn the most under conditions wherein previously identified hard and easy factors are both at play.

Overwhelmingly, faculty support was the highest explanation for why students felt they learned the most. Students define faculty support as having a course design or setup that helps facilitate their learning (e.g., scaffolding, peer instruction, welcoming/supportive learning community), which includes reducing their stress levels and otherwise being an “excellent” teacher. Student interest and perception of the utility/applicability of the material in the course also strongly contributed to student learning in both 100- and 300-level courses. Students recognize that increased cognitive load coupled with high faculty support (scaffolding and course design) contributes to their ability to learn and master challenging material.

DISCUSSION

Introductory Students Define Rigor Based on Effort/Workload

While academic rigor is important and highly valued, little research exists to define and explore it. This research took a sampling of introductory and advanced biology students and asked them to define academic rigor through the lens of the learner. Their definitions provide insight about what attributes of a course make it rigorous. For example, students identified hard/challenging/rigorous courses primarily in response to their own feelings of “drinking from a firehose.” They described these courses as ones that went too fast for them to keep up, for which they were not interested in the content or its relevance to their lives, and for which the workload was exceptionally high (Figure 2). This definition is consistent with what Draeger et al. (2015) found in their interview studies, specifically, that students emphasize workload elements (e.g., amount of work, number of assignments) over cognitive complexity. Interestingly, this definition appears to differ from faculty definitions of course rigor, which focus primarily on cognitive load (Michael, 2007; Draeger et al., 2015).

Upper-Level Students Define Rigor Based on Cognitive Demand

At the 300 level, students likewise identified cognitive demand as part of what makes a course intellectually rigorous. This was surprising, because other researchers who asked upper-level students to define rigor found no mention of “higher-order thinking” in their definitions (Draeger et al., 2015). Their upper-level students’ view of academic rigor, branching out from workload and effort, was more consistent with faculty definitions of academic rigor (Michael, 2007; Draeger et al., 2013, 2015). These findings suggests a learning progression related to academic rigor (Figure 8), whereby third- and fourth- year students are able to notice the cognitive challenge as distinct from the workload requirements.

FIGURE 8.

Progression of student perception of academic rigor. Student definitions showed a trend toward emphasizing cognitive complexity as an attribute of course rigor by the end of their college careers. Notably, academic stressors still remained a contributor to difficult courses; however, these upper-level students were also able to articulate that poor alignment between what was taught and assessed contributed to why a course was challenging.

Upper-level students also identified some pedagogical choices by the instructor as having an influence on the difficulty of the course. For example, they found courses that had low alignment between what was taught and assessed to be more difficult than courses that were well aligned. While these additional definitions of rigor are important to note, 300-level students still placed a premium on non–content focused course attributes (e.g., pace, stress level, workload) to define rigor, suggesting that the affective experience of a course influences perceptions of rigor, as does the cognitive experience.

In summary, while student definitions of academic rigor change as they develop into mature learners (Figure 8), they maintain perceptions of course rigor that are distinct from faculty definitions of academic rigor, which focus on engaging in higher-order thinking. While students identify and value high cognitive load, their perception of academic rigor is greatly colored by affective conditions—primarily stress—that accompany a learning experience. Academic stress and a coinciding perception of “difficulty” in our sample population are linked to unattainable pace, high workload and content volume, unfair assessment (due to lack of alignment; content appearing on exams that “was not covered” in class), disorganized course structure (lack of prioritizing or contextualizing content), lack of feedback, and feeling unsupported in the learning process (e.g., peer learning, relevant application of content, and instructor real-time feedback, active-learning strategies). Thus, the experienced “difficulty” or “rigor” for these students is more multidimensional—encompassing both affective and cognitive experiences of rigor—compared with faculty conceptualizations, focusing just on cognition.

Active-Learning Courses Are Perceived as Both Rigorous and Not Rigorous

While there is a wealth of research discussing the merits of active learning for student short- and long-term content retention (Freeman et al., 2014), to our knowledge, no studies looked at how students perceived the learning in these courses from the perspective of academic rigor.

Active learning interacts with a student’s definition of academic rigor. When asked about whether or not their active-learning course was easy or hard, nearly 50% of the students sampled identified their active-learning course as both easy and hard, and the remaining students were almost equally divided among the course being easy (24%) and hard (29%). These findings indicate that active-learning courses may be viewed as easy while simultaneously being rigorous (i.e., “learning meaningful content with higher-order thinking at the appropriate level of expectation in a given context” [Draeger et al., 2013, p. 268], leading to ultimate ownership of one’s learning [Bain, 2004]). Three of the four attributes of our definition of rigor aligned with the ways students defined “hard,” yet the fourth fell on the easy side.

Learners Engage in Higher-Level Cognitive Processes.

Interestingly, at both the 100 and 300 levels, students were able to parse out rigor when reflecting on their experiences in an active-learning classroom (Figure 5). Overwhelmingly, students identified that their active-learning courses had high cognitive challenge/demand. That is, they saw the courses were asking them to do more than recall and understand content. They had to “solve problems,” “analyze data,” “think critically,” and “think about real-world challenges.” When students defined hard courses in general (Figure 2), cognitive demand was frequently mentioned by 300-level students. At the introductory level, hard courses were more likely to be defined by academic stressors: high workload, quick pace, and unclear import. We maintain that an early (100-level) active-learning experience in biology could help students differentiate academic stress from cognitive complexity, especially if coupled with metacognitive reflection.

Learners Transfer Concepts and Content across Scales or between Problems.

Active learning, in this context, helped students transfer concepts, and students recognized this component of academic rigor in these courses. In their definitions, students identified that the case-based nature of these active-learning courses required them to transfer concepts between problems. For example, one student wrote,

“The concepts themselves were relatively easy to learn, but applying them to new situations added an aspect of challenge. This challenge was good, because it made me think in new ways, and helped the information ‘stick’ better.”

Learners Engage in Meaningful Content.

Students in these active-learning courses also elaborated on the real-life cases they studied as a part of the course design in their active-learning courses, identifying that they were engaging in the rigorous practice of practical application:

“It was much more beneficial to be able to apply the things we learned to actual life situations rather than multiple-­choice questions.”

Learners Have Appropriate Levels of Challenge and Support.

The fourth attribute of the definition of rigor is distinct from the other three in that students most often identified this attribute under the easy aspect of the active-learning courses. Many students were quick to articulate that their active-learning courses also felt easy/less rigorous to them because of the design of the courses. They state,

“I wouldn’t say that the content was easy—but the teaching style used to help me construct information was very useful, and as a result I retained information quite easily.”

“[Biology course] was an easy course for biology. It was easy because difficult concepts SEEMED easy.  The course was very structured so I knew what to expect—the learning objectives because this kept me on track (What do I need to know? What am I going to learn?). In addition, the modules make sense and are applicable to today’s societal ecological issues (e.g., learning microbiology in order to understand evolution and resistant bacteria, as well as other concepts). This makes it easier to learn because it makes sense and the puzzles come together at the end of each module.”

“It was an easy course for me because all the materials that I needed to learn were easily accessible to me. However, just because I consider the course to be easy, does not mean that I was able to slide through the class. It was important to me to utilize resources and make an effort to identify and master concepts. If no effort is made in the class, it would have been very difficult. However, all the materials are available for success.”

“What made the class easier was taking a hands-on approach and applying the concepts we learned.”

“The class was set up in a way that information built upon itself and was connected to things we had been learning throughout the entire semester. Making these connections made the information seem more simple.”

These courses appeared to have the necessary support—structures, class organization, faculty availability, active instruction—that contributed to decreased stress levels in the classroom, and thus the feeling of “ease.” This ease should be expected, based on their earlier definitions of academic rigor/challenge that overemphasized academic stressors as what makes a course difficult (Figure 2). In alignment with their definitions of easy courses (Figure 3), that such courses have a manageable workload and high support from faculty, students described their active-learning experiences as having high support. Therefore, it is not entirely surprising that they defined these courses as easy. Active-learning courses often draw students into content they might not otherwise have inherent interest in through problem-based learning or case studies. In addition, active-learning classrooms employ pedagogies that support student development in the learning process. These often include student-centered learning that puts the students in an active role for their learning—summarizing, investigating, modeling, revising, solving, and collaborating to achieve learning outcomes. Courses designed in this way by backward design are often well aligned with learning outcomes, assessments, and pedagogical choices. Thus, students are engaged in a supportive learning environment with low academic stressors (as they define it) dedicated to their success as learners. Through intentional “faculty support,” students transcend learning levels through the joy of discovery and problem solving as they make stepwise progressions through the content to gain new knowledge and use science process skills.

It is important to note that these same students identified the active-learning courses as hard/rigorous/challenging based on the other three attributes of the definition, yet seemed to give more weight to the fourth dimension of rigor, classifying their course experiences as both easy and hard more often than hard. Often, students commented that they did not “feel like they were learning” because of the way the material was presented or the way they engaged with the ideas. This made a course feel easy to the learner. Yet they also identified the courses as simultaneously being cognitively rigorous. This is an interesting dichotomy, one worth further investigation, especially as it departs from faculty definitions of academic rigor.

Implications of Rigor in Active Learning in Biology

Academically, we define rigor as “learning meaningful content with higher-order thinking at the appropriate level of expectation in a given context” (Draeger et al., 2013, p. 268), leading to ultimate ownership of one’s learning (Bain, 2004). Students desire such learning and are interested in courses that push and challenge them, with outcomes that are attainable through appropriate support (Martin et al., 2008). This desire for an appropriate balance of challenge and support (Sanford, 1962) indicates that students are not opposing a high workload or looking for an easy way out (Martin et al., 2008). Rather, they are looking for higher-order thinking on content that matters and a low-stress environment that enables all students to engage in learning, a “just-right” or “Goldilocks” level of challenge and support (Gordon and Palmon, 2010).

Students in this study were more likely to define a difficult or rigorous course in terms of both intellectual challenge and in terms of attainability, a finding that diverges from earlier work (Draeger et al., 2015). Conversely, when students discover the pace of a course to be too fast, or the workload to be too high with little support for the learner, they label the course as hard/rigorous/difficult. This finding is consistent with Nelson (2000), who said that academic rigor divided by faculty support provided equals course difficulty (rigor/support = difficulty). That is, when students are provided with a high degree of support in their learning, perceived course difficulty decreases, likely because the academic stressors decrease. We saw this come into play when students discussed how active learning influenced their perceptions of rigor, and how academically rigorous tasks felt easy to them (e.g., “I felt like I wasn’t learning”) because of this support.

Faculty surveyed in physiology did not identify the critical role that faculty have in contributing to the ease with which a student learns disciplinary content (Michael, 2007). In fact, faculty articulated it was solely the nature of the discipline and what students bring to the course that made physiology hard to learn (Michael, 2007). Students, however, were quick to acknowledge that not only their role in the learning process but that of the faculty member could contribute to the challenge/difficulty of learning a discipline (Michael, 2007). This research indicates the important role of a faculty member in the learning process of our biology students. Through active instruction, faculty can not only engage students in rigorous course work, but they can provide the support students need, which ultimately makes biology and STEM more accessible and inclusive for a greater diversity of learners.

Limitations

We recognize that the definition of rigor and its association with the terms “easy” and “hard” poses an interesting challenge for our measurements in this study. We identify that this is partly due to the fact that rigor is challenging to define. Because there are no validated instruments that measure the construct of rigor, we used terminology from prior studies that explored rigor—studies in which students described rigor as “hard,” “challenging,” “difficult”—and then we asked, “Why do students perceive their learning experience in this way?”  These studies found that the very use of the word “rigor” was challenging for students to understand in its own right (Draeger et al., 2015). Therefore, we decided to use the words “easy” and “hard” (based on how Michael [2007] posed the question to faculty) to elucidate student thinking about this aspect of rigor.  We designed our survey using language and terminology most relatable to students, so as to better understand their experiences in the classroom. However, we recognize that these findings isolate a subset of the construct of academic rigor.

Pedagogical Considerations

There are some important pedagogical and course design implications from this work. First, students want intellectual challenge (Gordon and Palmon, 2010). As instructors design learning outcomes for a course, they should plan to challenge student thinking and abilities, to push students to achieve beyond what they think is possible. Courses should have learning outcomes that span the cognitive domains (Bloom and Krathwohl, 1956; Lemons and Lemons, 2013), with an appropriate percentage of a course focusing on higher-order cognitive processes. Some professional societies for subdisciplines of biology, for example, have recommendations for learning outcomes across these scales that could serve as a starting place to help identify outcomes relevant to courses (American Association for the Advancement of Science, 2011). The Blooming Biology Tool (Crowe et al., 2008) can provide instructors with a concrete way to evaluate the level of intellectual processing their learning goals ask students to work toward. This tool can be used to explore one’s course work and exams as well as course objectives.

Second, students want intellectual challenge to be attainable. They do not want standards lowered, but they need the standard to be reachable through appropriate scaffolding. Identifying what is realistic for each class of students is challenging, as it likely is always changing based on the needs and preparation of changing learners. However, knowing one’s student population and their incoming knowledge and skill level will help instructors set the appropriate level of academic challenge. Conversations with faculty members who teach the preceding course(s) and pretests or concept inventories for each unit of instruction can help inform instructors of incoming students’ prior knowledge. In addition, once the course is in process, continual monitoring of student progress through frequent formative assessments, shared student reflection, and direct feedback from students can provide insight into the appropriateness of the level of challenge. We recognize that intellectual challenge will be experienced differently by each learner in the classroom. However, researchers in K–12 classrooms have a wealth of literature focused on differentiation that could be gleaned and applied to the college classroom. Martin et al. (2008) suggest something as simple as giving hints to students feeling overwhelmed or extension tasks/problems can foster an appropriate level of challenge for students on both ends of the spectrum.

Finally, students may need support in their learning. Active-learning pedagogies are designed, by their very nature, to provide support, including progressive steps to help students learn and master the content and skills needed to achieve course learning outcomes. These strategies include frequent formative assessments and cooperative learning approaches, which allow faculty members and peers to support the progress of learning through feedback and direction. In addition, active learning encourages active reflection; metacognition that helps students discover what it means to learn, how they learn, and how they are doing making progress toward learning outcomes. Peer and instructor support is often very high in active-learning environments, leading to safe learning spaces for students, which together lower stress and might incline students to say a course is easy. This study helps us to understand that easy reflects a more affective construct, such as low stress and ease. When students were challenged to learn at an attainable level, they felt the learning along the way was “intuitive”—they did not “feel like they were learning.” Upon reflection, students identified that these courses were some of the very ones in which they learned the most and that they recall more from these courses than others (Freeman et al., 2014).

These conclusions are in line with those of other researchers who identified the need to have appropriate challenge and support (Sanford, 1962) so that our courses are hard, but not too hard (Martin et al., 2008; Gordon and Palmon, 2010). Designing a course with challenge and support in mind requires intentional instructional decisions throughout the backward design process (Wiggins and McTighe, 1998), frequent reflective practice on the behalf of the instructor, and a willingness to try new pedagogies that can provide needed support without compromising academic challenge. In conclusion, when instructors preserve scientific rigor in their classrooms through high cognitive load (challenge), while delivering these courses in a learner-oriented manner (support), students experience intellectually rich learning without unnecessary stress or barriers to attainment.