A comparison of student performance and satisfaction between a traditional and integrative approach to teaching an introductory radiology course on the extremities

Abstract

Objective:

The purpose of the study was to compare student performance and student satisfaction ratings for an introductory extremities radiology course taught using 2 different educational methods.

Methods:

One group of students was taught using a traditional face-to-face instruction method, and the other group received an integrative blended-learning approach. A multivariate analysis of scores on lecture and laboratory examinations was performed to detect differences in student performance between the 2 methods. An independent t test was performed to compare the final course averages between the 2 methods. χ² Analysis was used to compare the distribution of letter grades and levels of satisfaction between the 2 groups.

Results:

Test scores were higher for the integrative approach than for the traditional face-to-face method (p < .05). However, the differences were not meaningful, as the greatest improvement in correct responses was only for 2 questions. Students appeared to be more satisfied with the integrative approach when compared to the traditional method (p < .05).

Conclusion:

Student satisfaction with the educational delivery methods in an introductory extremities radiology course using an integrative approach was greater than for the traditional face-to-face instruction method. Student performance was similar between the 2 cohorts.

INTRODUCTION

A traditional method of radiology instruction includes face-to-face lecture for 1 hour and a 1-hour laboratory session for students to view images in groups to solidify their understanding and interpretation of normal radiographic anatomy. However, several published studies support the effectiveness of a blended-learning, or integrative, approach to teaching radiographic anatomy by incorporating traditional learning techniques with electronic resources.^1–7 For example, Howlett et al.² investigated the use of various online educational techniques with traditional face-to-face teaching delivery and concluded that online learning offered the student access to a vast array of information that can be easily disseminated in contrast to traditional face-to-face techniques. Tam et al.⁷ concluded that computer-assisted learning can be a useful tool if well designed and integrated into current anatomy teaching methods and the curriculum. Shaffer and Small⁵ noted that computer-aided blended-learning tools effectively facilitated learning of radiology by increasing access to laboratory course materials. These studies provide evidence that applying techniques from blended-learning environments into radiographic anatomy courses provide adequate resources for prelaboratory preparation and can facilitate students' independent learning skills.

The reported restructuring of a 14-week radiology physics course⁸ was the model used for the present paper. In brief, the restructured radiology physics course consisted of 3 weeks of electronic learning (e-learning) instead of 4 weeks of traditional face-to-face instruction to introduce concepts on radiologic physics. The remaining 10 weeks of the course were committed to a blended-learning format with small group interactions and problem-based learning scenarios. The authors concluded that computer-based learning and course assessments embraced technology that is rapidly becoming an integral part of the teaching of anatomical radiology and clinical radiology. Furthermore, e-learning facilitated students' knowledge and comprehension of radiology physics, and there was a higher level of student satisfaction with the use of a learning management system (LMS).⁸

The purpose of the study was to compare student performance and student satisfaction ratings in an introductory extremities radiology course within a doctor of chiropractic (DC) program using 2 different educational methods (traditional method vs integrative approach). The 2 educational hypotheses were as follows: (1) Student grades on midterm and final course examinations would be higher with the integrative approach vs the traditional method, and (2) student satisfaction ratings on course evaluation questions assessing methods to facilitate student learning would be higher with the integrative approach vs the traditional method. If these educational hypotheses on e-learning were not supported within the context of teaching extremities radiology to DC students, then the instructor was willing to reconsider teaching the course using the traditional method.

METHODS

Context

An institutional-wide policy to provide students with more academic free time by reducing the number of credit hours in the DC program occurred in fall 2000. The curriculum revision process required faculty members to review course hours and identify overlap of educational content between courses to make the most appropriate curricular decisions that maintained academic rigor for earning the DC degree. The revised curriculum began with the fall 2004 trimester. Ongoing curriculum reviews by faculty for content validity and outcomes data (eg, National Board scores, course averages, etc) were used to ensure horizontal and vertical integration of the revised curriculum.

With the implementation of the new curriculum, the extremities radiology course transitioned from 2 credit hours of lecture and laboratory instruction to 1 credit hour of laboratory instruction. From September 2004 to August 2011, the 1-hour credit course continued to use the traditional teacher-centered instruction, in which the instructor briefly introduced the regional anatomy topic in 10–20 minutes, with the remainder of the time being used by students to work in small groups at the view boxes to identify and interpret normal radiographic anatomy.

During the ongoing curriculum review from September 2009 to August 2011, the instructor of the extremities radiology course began to explore e-learning since previous studies concluded that e-learning was an effective educational tool when used in conjunction with a user-friendly LMS.^8–11 Key findings during this exploration phase that resonated with the instructor were that (1) students perceived that the computer-assisted model provided better quality of instruction than did the traditional view box teaching format;⁵ (2) students became less engaged in their learning environment when crowded around view boxes,⁵ and (3) blended-learning approaches improved prelaboratory preparation and facilitated student learning.^1–7 Simultaneously with the instructor's exploration of e-learning, the institution began to offer online programs that provided the instructor and students with access to a user-friendly LMS.

In accordance with the blended-learning methods for teaching radiologic anatomy in medical education,^1,12,13 the instructor introduced computer-aided learning tools for chiropractic students enrolled in the extremities radiology course beginning in September 2011. This revised educational delivery method for the extremities radiology course was referred to as an integrative approach because of the combined use of e-learning and classroom instruction. A description of the development and evaluation of the computer-aided learning tools has been published elsewhere, including the use of student satisfaction measures.^14,15 Although the reliability and validity of student evaluations of teaching to measure teaching effectiveness is a topic of much debate, student ratings on course evaluations do provide reliable and valid information on student satisfaction with the learning environment.^16–25

Design and Participants

The experimental design was a cross-sectional comparison of 2 educational delivery methods. Participants were 2 cohorts of students from different academic years: September 2010 to August 2011 and September 2011 to August 2012. The academic year that students were enrolled in the extremities radiology course determined their cohort assignment to the traditional method (n =184, 2010–2011) or their cohort assignment to the integrative approach (n = 178, 2011–2012). Each academic year cohort included students enrolled in fall, winter, and spring trimesters, that is, 3 trimester cohorts of students per educational delivery method cohort. This research was approved by the institutional review board of New York Chiropractic College.

Educational Delivery Methods

The traditional method involved a brief introduction of the regional anatomy by the instructor, with the remaining class time being structured to allow students to work in small groups at the view boxes to identify and interpret radiographic anatomy. Beyond class notes and textbooks, the only learning aid was a student workbook with homework assignments for each anatomical region.

The integrative approach included the same face-to-face brief laboratory introduction as with the traditional method, view box learning activities, and homework assignments, but it also included computer-aided learning tools. The computer-aided learning tools included interactive digital radiography modules for identification of normal anatomy, mensuration procedures, case studies, and computer-based self-assessment tutorials that were available with 24-hour access to all students until the end of the course. Interactive multiple-choice questions, Jeopardy-style games, and Hollywood Squares-style games were examples of computer-aided learning tools that were included in the classroom sessions as instructor-guided computer-assisted classroom activities to break up the repetitiveness of the weekly view box learning approach.^14,26 Students participated in e-learning activities during class time using desktop computers at students' desks. Access to the computer-aided learning tools was through the LMS, Desire-2-Learn (D2L Corporation, Kitchener, ON, Canada).

The duration and amount of instruction and the depth of topic content was the same for both educational delivery methods. The in-class teaching strategy followed the traditional method of laboratory instruction. Both cohorts had instructional delivery for 12 sessions with 2 additional sessions reserved for examinations. The only difference between the 2 educational delivery methods was the integration of computer-aided learning tools into classroom activities and as study resources. Computer-aided learning tools were designed to provide adequate study resources to students to facilitate their understanding and interpretation of radiographic anatomy, self-learning skills, and prelaboratory preparation. Both cohorts were taught by the same instructor and used the same textbooks and supplemental materials. However, the teaching assistants were not the same individuals across trimester cohorts within and between the educational delivery methods. Course syllabi described the details of the traditional method and integrative approach.¹⁴

Outcome Measures

Student performance outcomes on midterm and final course examinations measured knowledge, comprehension, application, and synthesis of course content taught in extremities radiology. Student performance outcomes were obtained from deidentified grade book records. Final grade averages and distribution of letter grades were also recorded by educational delivery method. The instructor used the same examinations in both lecture and laboratory testing situations for all trimester cohorts, that is, fall, winter, spring trimester cohorts for the traditional method and again for fall, winter, and spring trimester cohorts for the integrative approach.

Student satisfaction was measured by frequencies of responses to the following questions from the institutional-based course evaluation system: (1) Organization of the faculty member; (2) Faculty member encouraged students to participate in class; (3) Use of class time supported student learning; (4) Class resources and instruction contributed to student learning; and (5) Effective use of class time by faculty member. The frequency of responses to the following 2 questions from the institutional-based course evaluation system were used to account for the potential confounding factors of course workload and student self-motivation: (1) students completed course assignments and readings as assigned, and (2) students allocated sufficient time for study, respectively. The anonymous course evaluations data were obtained from electronic records stored by the faculty secretary assigned this responsibility. The course evaluation questions were rated on 5-point Likert scale: 1, strongly disagree; 2, disagree; 3,neutral; 4, agree, and 5, strongly agree. Course evaluations were administered during weeks 10 through 12 of the trimester.

As attendance is a potential confounder variable affecting primary outcomes of student performance and student satisfaction, there was a comparison of attendance records between cohorts of students assigned to each of the educational delivery methods, that is, traditional method vs integrative approach.

Data Management and Statistical Analysis

Deidentified grade rosters containing examination scores and final course averages were exported into Excel (Microsoft, Redmond, WA) spreadsheets. The data were coded by trimester cohort and educational delivery method. The frequencies of responses from course evaluation questions were exported into Excel spreadsheets by trimester cohort and educational delivery method. The Excel spreadsheets were imported into SPSS Statistics for Windows, (version 21.0; IBM Corp., Armonk, NY) for statistical analysis. A multivariate analysis of scores on lecture and laboratory examinations as a function of educational delivery method and examination time point, midterm or final, was performed using a 2 × 2 multivariate analysis of covariance model with repeated measures on examination time point to detect differences in student performance between the educational delivery methods. An independent t test was performed to compare the final course averages between the traditional method and integrative approach. A χ² analysis, 2 × 4 contingency table, was performed to compare the distribution of letter grades between the 2 educational delivery methods.

χ² Analyses, 2 × 5 contingency tables, were performed for each course evaluation question to detect changes in the distribution of student satisfaction ratings as a function of the educational delivery method. These χ² analyses were also performed on the 2 student evaluation questions that addressed the potential confounding factors of course workload and student self-motivation on course performance and satisfaction ratings. Secondary χ² analyses, 3 × 5 contingency tables, were performed for each course evaluation question to detect any changes in Likert scale ratings across trimester cohorts within each educational delivery method.

RESULTS

In-class attendance was 95% or greater throughout the trimesters for both cohorts of students. All students that were present for class participated in the learning activities designed for their cohort after a brief topic introduction. The cohort of students in the traditional method completed the view box assignments. The cohort of students in the integrative approach completed the view box assignments and participated in the instructor-guided computer-assisted classroom activities. In addition to classroom learning activities, the cohort of students in the integrative approach used the D2L LMS to access the computer-aided learning tools as study resources outside of regular classroom time. No comments were posted about technical difficulties in the forums of the D2L LMS.

A total of 362 students completed the radiology course with a similar distribution between the traditional method (n = 184; students in fall [n = 42], winter [n = 22)], and spring [n = 120] trimesters) and integrative approach (n = 178; students in fall [n = 35] winter [n = 24], and spring [n = 119] trimesters). However, only 263 students completed course evaluations. In addition, the distributions of students completing course evaluations were disproportionate between the 2 educational delivery methods by trimester. Eighty-three percent to 93% of the students completed course evaluations in the fall 2010 trimester, traditional method (35–39 recorded responses per questions), while only 54%– 65% of students completed course evaluations in the fall 2011 trimester, integrative approach (19–23 recorded responses per questions). More than twice as many students completed course evaluations in the spring 2012 trimester, integrative approach (108–110 recorded responses per questions, ≈ 92% participation), as compared to the spring 2011 trimester, traditional methods (46–51 responses recorded per questions, ≈43% participation).

The overall test scores were slightly higher for the integrative approach than for the traditional method (multivariate Wilks Λ F_{(2, 359) group} = 7.26, p < .05). However, the differences were not meaningful as the greatest improvements in correct responses were only for 2 questions, as revealed for the lecture midterm (F_{[1, 360] group × Exam_Period} = 56.18, p < .05). The comparison of interquartile ranges from box plots for each examination between the traditional and integrative instructional methods suggested greater consistency in learning among the cohort of students receiving the integrative approach. The final distributions of letter grades earned by the students were similar for the traditional method and the integrative approach, and the final numeric grades by educational delivery methods were equivalent to a final letter grade of A (Table 1, p > .05).

Table 1.

Letter Grade Distribution

Instructional Method	Final Letter Grade Distribution, n (%)
	A	B	C	F	Final Grade Averagea
Traditional	113 (61.4%)	61 (33.2%)	9 (4.9%)	1 (0.5%)	90.0 ± 6.04
Integrative	126 (70.8%)	47 (26.4%)	5 (2.8%)	0 (0.0%)	91.3 ± 4.80

^aFinal grade average is mean ± SD.

Within the completed course evaluations, there were some missing responses for questions. Any bias in the student evaluations is difficult to address statistically as the confounder is missing respondents, and there is no theoretical framework to predict the ratings of these missing respondents. Thus, χ² analyses of student satisfaction ratings were based on 99–111 respondents per question from the traditional cohort (≈57% response rate) and 145–153 respondents per question from the integrative cohort (≈84% response rate).

Students in both cohorts indicated that they completed the course assignments and readings as assigned (Table 2; χ²[df = 3, n = 263] = 7.58, p = .056). There was a shift in the ratings on students allocating sufficient time for study toward strongly agreeing with the integrative approach as compared to the traditional method (Table 2; χ²[df = 4, n = 258] = 16.81, p < .05). However, based on the secondary χ² analyses, there was an adaptation period from traditional method to the integrative approach. Students in the fall cohort 2 and winter cohort 2 integrative trimesters reported that it was more difficult to complete course assignments and readings as assigned than did students in the spring cohort 2 integrative trimester (Table 3; χ²[df = 6, n = 152] = 20.69, p < .05). This was indicated by the greater proportions of ratings in the categories of disagree and neutral from students in the fall cohort 2 and winter cohort 2 integrative trimesters than in spring cohort 2 integrative trimester. The distribution of students allocating sufficient time for study also shifted toward strongly agree after the fall cohort 2 integrative trimester (Table 3; χ²[df = 8, n = 152] = 16.74, p < .05), as indicated by the greater proportions of ratings in the category of strongly agree from students in winter cohort 2 and spring cohort 2 integrative trimesters than in the fall cohort 2 integrative trimester.

Table 2.

Distribution of Student Ratings by Instructional Method (Count and % Within Group)

Question	Instructional Method	Ratingsa
		1	2	3	4	5
Students were able to complete the course assignments and readings
	Traditional	0 (0%)	1 (0.9%)	8 (7.2%)	42 (37.8%)	60 (54.1%)
	Integrative	0 (0%)	2 (1.3%)	9 (5.9%)	35 (23.0%)	106 (69.7%)
Students allocated sufficient time for study
	Traditional	0 (0.0%)	0 (0.0%)	12 (11.3%)	55 (51.9%)	39 (36.8%)
	Integrative	1 (0.7%)	1 (0.7%)	8 (5.3%)	50 (32.9%)	92 (60.5%)
Organization of faculty member
	Traditional	2 (1.9%)	7 (6.6%)	16 (15.1%)	40 (37.7%)	41 (38.7%)
	Integrative	0 (0.0%)	0 (0.0%)	10 (6.6%)	51 (33.8%)	90 (59.6%)
Faculty members encouraged students to participate in class
	Traditional	1 (0.9%)	2 (1.9%)	16 (15.1%)	44 (41.5%)	43 (40.6%)
	Integrative	0 (0.0%)	0 (0.0%)	7 (4.6%)	41 (27.0%)	104 (68.4%)
Use of class time supported student learning
	Traditional	4 (3.8%)	6 (5.8%)	14 (13.5%)	44 (42.3%)	36 (34.6%)
	Integrative	1 (0.7%)	3 (2.0%)	11 (7.5%)	42 (28.6%)	90 (61.2%)
Class resources and instruction contributed to student learning
	Traditional	2 (1.8%)	5 (4.6%)	17 (15.6%)	49 (45.0%)	36 (33.0%)
	Integrative	0 (0.0%)	1 (0.7%)	19 (12.6%)	42 (27.8%)	89 (58.9%)
Effective use of class time by faculty member
	Traditional	2 (2.1%)	7 (7.3%)	10 (10.4%)	41 (42.7%)	36 (37.5%)
	Integrative	1 (0.8%)	0 (0.0%)	7 (5.6%)	35 (28.0%)	82 (65.6%)

^aScale for ratings: 1 = Strongly Disagree, 2 = Disagree, 3 = Neutral, 4 = Agree, 5 = Strongly Agree.

Table 3.

Integrative Approach by Trimester (Count and % Within Group)

		Ratingsa
Question	Integrative Trimester	1	2	3	4	5
Students were able to complete the course assignments and readings
	Fall 2011	0 (0.0%)	2 (8.7%)	1 (4.3%)	3 (13.0%)	17 (73.9%)
	Winter 2012	0 (0.0%)	0 (0.0%)	4 (20.0%)	4 (20.0%)	12 (60.0%)
	Spring 2012	0 (0.0%)	0 (0.0%)	4 (3.7%)	28 (25.7%)	77 (70.6%)
Students allocated sufficient time for study
	Fall 2011	1 (4.3%)	0 (0.0%)	0 (0.0%)	10 (43.5%)	12 (52.2%)
	Winter 2012	0 (0.0%)	1 (5.3%)	2 (10.5%)	4 (21.1%)	12 (63.2%)
	Spring 2012	0 (0.0%)	0 (0.0%)	6 (5.5%)	36 (32.7%)	68 (61.8%)

^aScale for ratings: 1 = Strongly Disagree, 2 = Disagree, 3 = Neutral, 4 = Agree, 5 = Strongly Agree.

The integrative approach shifted the distribution of student satisfaction ratings toward strongly agree as compared to the traditional method (Table 2, p < .05): (1) organization of the faculty member (χ²[df = 4, n = 257] = 22.86); (2) faculty member encouraged students to participate in class (χ²[df = 4, n = 258) = 24.52]; (3) use of class time supported student learning (χ²[df = 4, n = 251] = 19.56); (4) class resources and instruction contributed to student learning ( χ²[df = 4, n = 260] = 21.57); and (5) effective use of class time by faculty member (χ²[df = 4, n = 221] = 22.86). Secondary χ² analyses revealed that student satisfaction ratings for these 5 questions were consistent among the 3 trimesters within each cohort (p > .05). Collectively, the similar responses related to student satisfaction ratings among the 3 trimester cohorts within each educational delivery method suggested that bias from the missing respondents may not have confounded the student evaluation data.

DISCUSSION

The primary purpose of this study was to compare 2 teaching delivery methods, face-to-face (traditional method) and computer-aided blended approach (integrative approach), with respect to student performance and student satisfaction. This study found that there was no significant difference between the 2 teaching methods with regard to student performance. However, students perceived that the integrative approach facilitated their learning of the course material as compared to the traditional method because they were more satisfied with the integrative approach as an educational delivery method. The current research extended insights on e-learning and education in radiology for medical students to include a promising blended-learning approach to educate chiropractic students in radiology.

Although student performance was similar between the 2 educational delivery methods, the integrative approach resulted in more consistent student performance than the traditional method. The literature is equivocal when describing the effects of educational delivery methods on student performance. Bain²⁷ found that blended-learning increased student performance on assignments that required critical thinking skills, written communications, and advanced course objectives, but group projects and final grades were similar between blended-learning sections and traditional face-to-face sections. Course assessment grades and final grades were better for students enrolled in the online section versus the lecture section of an introductory computer programming course.²⁸ Lopez-Perez et al.²⁹ concluded that a blended-learning approach improved examination scores in university courses, but the impact of the educational delivery method on student performance was co-related to the types of blended-learning activities and on students' age, background, and class attendance rate. Taradi et al.³⁰ reported that student performance in a physiology course taught using computer-aided learning tools with in-class face-to-face problem-based instruction was more consistent than when the information was taught using only in-class face-to-face problem-based instruction. The study by Taradi et al.³⁰ was similar to the current study in that assignment to educational delivery method was based on the academic year that second year undergraduate medical students enrolled in the course and that computer-aided learning tools were integrated into the classroom instructional method. Similarly, the evidence-based literature in radiology demonstrated that the effects of educational delivery methods on student learning were inconsistent and required additional research.^{1,12,13,31–34}

In the current study, the final distributions of letter grades earned by the students were similar for both educational delivery methods. Laumakis et al.³⁵ and Larson and Sung³⁶ reported similar findings. Conger³⁷ referenced that numerous variables should be considered when assessing the influence of different styles of course delivery on student performance, including the following: learner characteristics, instructor/student interaction, instructional design, instructional activities/type of course materials, and media attributes. Larson and Sung³⁶ postulated that students will adapt and learn by whatever medium is available to them. However, Pinto et al.,⁴ Gotthardt et al.,⁸ and Picciano,³⁸ relate that there is a new generation of learners that are technically fluent, and as a result, teachers need to make adjustments to their educational delivery methods that go beyond traditional paper-based teaching materials to include multimedia applications. Given that all students had instruction in the basic concepts of conventional radiography and anatomical structures prior to enrolling in the course and were taking the extremities radiology course for the first time, one may speculate that students had a solid background in the basic anatomical sciences, which translated into both cohorts achieving similar final course averages. The more consistent performance among the students receiving the integrative approach may suggest a beneficial learning effect of this educational delivery method.

The current study also addressed student satisfaction using trimester institutional-based standard course evaluations. The students enrolled in the integrative approach were more satisfied in comparison to their counterparts in the traditional method. The distributions of ratings shifted toward strongly agree for the integrative approach as compared to the traditional method. Previous educational research indicated that ratings of student satisfaction were similar among face-to-face, blended, and online delivery methods.^35,36 However, Sitter et al.³⁹ reported that both students and faculty perceived a blended-learning approach to be an effective educational delivery method for courses. Positive attitudes toward e-learning by students in radiology education was a consistent finding in the literature.^{1,8,12,13,33,34} In addition, having 24-hour access to the computer-aided learning tools may underlie greater student satisfaction with a blended-learning approach as found in the current study and previous research by providing students with more options to allocate their study time and greater access to study resources.

Computer-assisted learning should engage the learner if it is to serve its function.⁴⁰ A vast array of blended-learning tools needs to be incorporated into the course design to enhance a sense of community as well as learning and satisfaction rates.^41,42 Several researchers found a significantly higher motivation and satisfaction rate among learners who were exposed to blended learning.^41,43–46 Picciano³⁸ concluded that in a classroom there are different types of learners based on age, learning styles, and personalities, and therefore courses should be designed to cater to these facets by incorporating multiple techniques into the blended-learning environment. This dynamic approach allows students to experience different avenues of learning, which may be beneficial to them.³⁸ As many students transition themselves away from a textbook to internet resources, a blended teaching platform is an acceptable learning vehicle for them and is almost expected.^4,47,48

Increased student satisfaction with the integrative approach was based on greater percentages of responses in the strongly agree category as compared to the traditional method. An outcome measured on a Likert scale is an ordinal variable, which is discrete in nature.⁴⁹ Likert scale items are representative of a multinomial distribution in which each response can be placed into 1 of n categories, and ordinal scales do not address how respondents discriminate between categories.⁴⁹ Thus, it was concluded that percentage differences in the strongly agree category between the 2 educational delivery methods represented a meaningful difference in student satisfaction ratings, and the χ² analysis of the 2 × 5 contingency table was the most appropriate statistical procedure to detect differences in student satisfaction ratings.

In summary, we offer the following explanations for greater student satisfaction for integrative approach vs the traditional method that was observed in the current study. During in-class interactive sessions, all students participated, appeared more focused, attentive, and enthusiastic. Motivating students and providing active learning activities promotes positive participation, stimulates the learning process, and provides immediate feedback for formative self-assessment. A well-established LMS with ease of use also provides an avenue to facilitate learning and increase student satisfaction. As mentioned previously, e-learning can be a very effective tool when used in conjunction with a user-friendly LMS.^8–11 Although workload was perceived as being greater during the first 2 trimesters of the transition to the integrative approach, there was no difference between the educational delivery methods in the content or the volume of materials presented. During the transition to the integrative approach, students may have perceived a greater workload as they were still adapting to the best way to utilize the computer-aided learning tools. The instructor may have also been adapting the guidance provided to the students for utilizing the computer-aided learning tools in class and as a study resource. Overall, the integrative approach shifted the distributions of scores for students allocating enough time to study toward strongly agree as compared to the traditional method. This may indicate that computer-aided learning tools facilitated student learning, that is, were deemed helpful by the students.

Limitations

The selection of subjects was not random. Random sampling is necessary to increase the precision of the study sample with respect to being more representative of the target population and minimizing the effects of subject selection bias on external validity. Cross-sectional designs may introduce information bias. The college mandates that the students cannot be absent more than 20% of class time, and attendance was 95% or greater in the current study. However, class absences may impact the number of students participating in some of the interactive sessions or filling out course evaluations, which may affect outcome measures. The motivation of students to perform on course examinations and the subjective nature of course evaluations are other sources of information bias. These sources of information bias may exist within and between the cohorts. Beyond their subjective nature, course evaluations were conducted approximately 2–3 weeks after the course midterm examination. At the time of course evaluations, students were still completing activities related to their homework grade and had not yet taken their final examinations. However, both cohorts had the same time line for reflection about the course, and therefore information bias related to student satisfaction should be equally biased.

In cross-sectional designs, another potential limitation may exist within the culture of the cohort. Instructor–student interactions may differ between cohorts due to the different educational and personality backgrounds of teaching assistants and the students themselves. In the current study, teaching assistants were not the same individuals across trimester cohorts within and between the educational delivery methods. Additionally, the lead instructor may have been more engaging with the integrative approach cohort as the educational delivery method was newly designed to facilitate student learning.

The number of students completing the course was similar from both cohorts, traditional method (n = 184) and integrative approach (n = 178), but the distributions of students completing course evaluations by trimester were disproportional between the 2 cohorts; 99–111 students from the traditional cohort (≈57%) and 145–153 students from the integrative cohort (≈84%). It is the college's policy that students are highly encouraged to participate in the standardized course evaluation process, but it is not mandatory. The similar responses related to student satisfaction among the 3 trimesters within each cohort suggested that bias from the missing respondents may not have confounded the student evaluation data. Furthermore, higher response rates may or may not lead to better accuracy in determining student satisfaction due to nonresponse bias.^50–52 However, the bias from the missing respondents on the student evaluation data is unknown.

There was no pre-post assessment of course content to infer greater learning as a function of educational delivery methods. The course evaluation questions did not directly assess student satisfaction with the educational delivery methods. Including student voice data collection methodologies into the current study may have provided more insights on the differences between the traditional method and the integrative approach with respect to student learning and student satisfaction.^31,53,54 Comparative instructional design studies are still needed to identify educational delivery methods that maximize learning in the field of radiology.^12,32

CONCLUSION

Student satisfaction with the educational delivery methods in an introductory extremities radiology course using an integrative approach was greater than for the traditional method. Student performance was similar between the 2 educational delivery methods.