Leveraging learning science to improve student outcomes in asynchronous online medical terminology education

ABSTRACT

Background

Online instruction within higher education is a growing trend. Asynchronous online courses vary widely in design elements. Research is needed to evaluate the impact of course design on student outcomes. A large, asynchronous undergraduate medical terminology course was modified to increase authentic language use, student interaction, formative feedback, retrieval practice, and metacognition. The purposes of this study were to describe modified course design elements and evaluate the impact of modifications on student outcomes compared with a standard course implemented concurrently.

Methods

This prospective, quasi-experimental study included 494 students (modified course, n = 277; standard course, n = 217). Measures included student participation, performance, course satisfaction, self-efficacy (SE), and engagement.

Results

Participation in assignments was high (88–94%). Students in both courses experienced growth in SE for medical terminology use. Students in the modified course earned significantly higher course grades and reported higher satisfaction levels. No significant differences in SE or exam scores were found between courses. Marginal significance was found for increased behavioral engagement for students in the modified course.

Conclusions

Modifications based on language learning and learning science principles were feasible to implement within a large, asynchronous online medical terminology course. Modifications resulted in greater student satisfaction and improved course grades. Exam performance was not significantly different between the modified and standard courses. Future research should focus on modifications preparing students for summative assessments.

Introduction

Online instruction within higher education was a growing trend even before the COVID-19 pandemic required a rapid shift from in-person to online learning [1]. Online courses may be delivered in real-time (synchronous), without real-time components (asynchronous), or as hybrid courses with both synchronous and asynchronous components [2]. Much of the increase in distance learning has been in the form of asynchronous online courses. Medical terminology is an example of a course that lends well to an asynchronous learning environment because the pedagogical nature of the content is foundational and factual [3]. The online teaching and learning environment presents unique challenges from traditional classroom settings for both students and instructors. For example, students may experience isolation in the absence of authentic student-to-student and student-to-instructor interactions. In addition to course content expertise, instructors must be knowledgeable in using effective asynchronous teaching strategies and manage technology [4,5]. As online course offerings become commonplace, educational research is needed to understand the relationship between online course design and student outcomes [3].

Medical terminology is the language of healthcare services, which includes many types of professionals. Medical terminology is a requirement for health-related education programs and is a prerequisite for professional and graduate programs in health-related fields. The result of high demand for medical terminology courses is large class sizes. Learning medical terminology has been compared with learning a second language, with similar relationships to learning strategies and student outcomes [6,7]. Methods for teaching medical terminology within online, asynchronous course designs must leverage language learning principles, be practical with large student-to-instructor ratios, and positively impact student learning behavior and outcomes [7].

Specific to learning language in an online context, students benefit from opportunities to engage in authentic language use, timely feedback, and social interaction [5,8]. Exposure to the authentic context in which medical terms are used fosters more proficient use of medical language [9]. Receiving feedback on language use promotes language learning in part because feedback promotes self-regulated learning, including monitoring and evaluating one’s own learning [8]. Time constraints on instructors in large online courses can limit the amount of individual feedback that instructors can provide [8,10]. Peer formative feedback may ease the time burden of using feedback as part of instruction and serve as a form of collaborative learning among students [8].

The relationship between online course design elements and student learning outcomes is not clear. Student achievement is commonly measured using graded assessments. Importantly, course characteristics may impact learning assessment design. For example, large online courses may be limited to quiz-like, multiple choice assessments due to feasibility of grading. Other measures, such as achievement emotions, self-efficacy (SE), and student engagement, may inform a more complete picture of a student’s learning experience. Achievement emotions may be positive (e.g., enjoyment) or negative (e.g., boredom) and are associated with achievement-related student behavior [11]. Self-efficacy (SE) is an individual’s belief in their capability to perform a specific task at a particular level [12]. A significant and positive relationship exists between academic performance and SE beliefs for second language learning [7,13–17]. Student engagement is the student’s psychological investment in and effort directed toward learning [18]. Increased student interaction may result in greater student achievement, possibly due to increased engagement [19–21].

At a large, public, midwestern university, medical terminology is an undergraduate course delivered in a completely asynchronous online learning environment. A course design modification was developed using strategies to intentionally (1) provide opportunities for authentic language use; (2) increase student-student interaction; (3) provide students with accurate and timely feedback; (4) increase student-instructor interaction; (5) create opportunities for retrieval practice; and (6) promote metacognition. The goal of these modifications was to positively impact student outcomes when compared to the standard course design. No changes were made to course content or assessments between the modified and standard course design. The purposes of this study were to (1) describe modified course design elements that employed language learning and learning science principles; (2) evaluate student participation in modified course design elements; and (3) measure the impact of course modifications on student performance, satisfaction, SE, and engagement.

Materials and methods

Ethics approval and consent to participate

The study protocol and procedures were reviewed and approved by The Ohio State University Institutional Review Board (OSU IRB: 2019B0354). Informed consent was obtained from all participants included in this study.

Research design and recruitment

All students enrolled in the online medical terminology course over two semesters were recruited for this prospective, quasi-experimental study using emails and course announcements. A total of 505 students provided consent to participate in this study. After consenting, 11 students from the modified course withdrew. Final analyses included 494 students, with 277 students from the modified course and 217 students from the standard course.

Description of modified course design elements

Promote authentic language use, increase peer-peer interaction, and provide accurate, timely feedback

Students in the standard course completed a weekly medical term pronunciation assignment using a digital learning tool (MindTap from Cengage Learning, 2016) [22] with a voice recognition feature. Students read a list of medical terms from weekly content. The software provided feedback regarding the accuracy of the student’s pronunciation. For the modified course, students self-recorded a reading of 1-minute case description focused on medical terms from that week’s content. Case description recordings were submitted to both the instructor and to a small group of 6–7 students. These small groups remained consistent throughout the semester. Three students within the small group were randomly assigned to provide feedback for each recording using a rubric developed for this assignment called the Confident Communication Feedback Form (Table 1). Five domains of confident communication were rated: pace, fluency, pronunciation, volume, and enunciation. These domains focused on modifiable qualities of language use, making the feedback objective and constructive.

Table 1.

Confident communication feedback form.

Confident Communication Domain	Brief Domain Definition	Rating Criteria
		Evidence of this domain:	Difficulty within this domain:
Pace	Speed of talking	When someone uses good pace, their talking is not too fast or too slow. The speed is just right so that a listener can easily hear each word but not become distracted with pauses that are too long.	Talking was either too fast or too slow, making listening more challenging.
Fluency	Rhythm of talking	When someone speaks fluently, they are talking without much effort. They use a steady rhythm in speaking.	The speaker hesitated or stumbled over words or had an unsteady rhythm in speaking, which made listening more difficult.
Pronunciation	Accuracy of talking	The speaker said the words in the passage correctly when compared to the gold standard recording.	Words were pronounced incorrectly when compared to the gold standard recording.
Volume	Sound level of talking	A good volume is not too soft, which would make it difficult for a listener to hear. It is also not too loud, which makes it uncomfortable to continue listening.	The speaker used a volume that was too loud or too soft, which made listening more difficult.
Enunciation	Clarity of talking	A speaker enunciates well when they say words clearly and plainly. A listener should not have to work hard to identify the words spoken.	The speaker did speak clearly and plainly, so that listening required increased effort.

The Confident Communication Feedback Form is the rubric designed for the pronunciation assignment in the modified course. Students used this rubric to provide confidential weekly peer feedback to three Communication Community group members.

Increase student-instructor interaction

For the modified course, student-instructor interaction was increased with a video introduction to the instructor, course, and syllabus; establishment of in-person office hours; and with an explicit email response policy to provide timely response to student messages. A weekly video message introduced new content and provided class-wide feedback to summarize performance on course assignments. These videos were brief (less than two minutes) and were recorded in a comfortable home setting using a conversational tone.

Create opportunities for retrieval practice

Students completed a total of nine formative assessments for each of five units in the course. Students in the standard course were given a three-week period to complete all nine formative assessments before the summative unit assessment. For the modified course, pacing for formative assessments was formalized so that students were required to complete three assignments weekly. Students in the modified course were provided up to three attempts per assignment to build in repeated retrieval practice.

Promote metacognition

The modified course included an optional module designed to promote metacognition. Content focused on effective learning strategies [23], levels of learning [24], recommendations for learning activities from previous students in the medical terminology course, and explicit connections between course assignments and learning outcomes for this course. Students completed a guided notes form while reviewing the pre-recorded content.

Measures

Student participation

Participation in the pronunciation assignments (for both courses) and optional metacognition assignment (for the modified course only) was measured by completion of assignments.

Student performance

Student performance was measured by the average of exam scores and overall course grade. Students had five summative examinations throughout the semester which accounted for 50% of the course grade. These exams covered three weeks of content and were not cumulative. Exams were administered and automatically graded within the course learning management system. Formative assessments also contributed to overall course grade. Exams and formative assessments were identical between the standard and modified courses.

Student Satisfaction Questionnaire

This questionnaire included 12 items related to student experience in completing the medical terminology course. Two items related to course assignments (usefulness and promoted learning); two addressed teaching methods (effectiveness and enjoyment); three were about course resources (ease of access, ease of use, and utilization of classmates as a resource); two items focused on communication (explicitly stated course expectations and clear grading criteria); and the final three items focused general perceptions of the course (course was intellectually stimulating, overall course satisfaction, and intent to recommend course to other students). Each item was scored on a 5-point Likert scale ranging from 1 (Strongly Disagree) to 5 (Strongly Agree). An overall satisfaction score was derived from the average of student ratings on this questionnaire.

Medical language self-efficacy (SE) scale

A medical language SE scale was developed for this study based on the structure of a valid and reliable measure of student self-efficacy in a different content domain [25]. The SE Scale included six items. Three items asked students to rate their confidence to understand medical language spoken by three different communication partners: a layperson, a peer (classmate/colleague), and a medical professional. The other three items asked students to rate their confidence in speaking medical language with the same three communication partners. For each item, students used an 11-point scale, ranging from 0 (‘Not at all confident’) to 10 (‘Extremely confident’). A pre-test and post-test SE score was calculated by averaging the six SE Scale items together for each time point.

Student Engagement in School Scale (SESS)

Student engagement was measured by the SESS [26]. The SESS is a student-report measure of engagement across three domains: affective, behavioral, and cognitive engagement. The Affective Engagement subscale includes nine items that measure how students feel about learning (e.g., ‘I like learning in school’). The Behavioral Engagement subscale includes 12 items that ask about student effort and persistence in learning (e.g., ‘I try hard to do well in school’). The Cognitive Engagement subscale consists of 12 items that measure how students process information for learning (e.g., ‘When I study, I try to connect what I am learning with my own experiences’). In total, the 33 items of the SESS are scored on a 5-point Likert scale ranging from 1 (Strongly Disagree/Never) to 5 (Strongly Agree/Always). The SESS has strong psychometric properties, including good test-retest reliability, internal consistency, and concurrent validity [26].

Procedures

When students enrolled in the medical terminology course, they were allocated to either the standard course or the modified course randomly and based on convenience by registrar administrators. Students were notified of the study by a course announcement and an email. Consenting students were directed to complete baseline outcome measures, including student self-reported demographic information (gender identity and class standing), the Medical Language SE scale, and the SESS. At the end of the course, students completed follow-up surveys, including the SE scale, the SESS, and the Student Satisfaction Questionnaire. Measures were completed online (QualtricsXM, Provo, UT). Exam grades and overall grades were extracted from the online learning management system. All information was de-identified prior to data analysis.

Data analysis

All assumptions were verified and, when necessary, outliers (scores >2.5 SD below the mean) were removed to achieve normality. Proportions within the modified and standard courses were calculated for student gender and class standing. Pearson’s chi-square test for independence was used to compare student gender and class standing between courses. Post hoc testing was done using adjusted Pearson residuals where z scores > 1.96 indicated a significant difference between courses. Proportion of students who completed the metacognition assignment was calculated for the modified course. A Kruskal-Wallis test was conducted to compare the percentage of completed assignments between courses. Related-samples Wilcoxon Signed Rank tests were used for comparing pre-test and post-test scores for the SE scale. Between-course differences in average exam score, final grade, and course satisfaction were assessed with a series of t-tests. Finally, an analysis of covariance (ANCOVA) was used to assess differences between the two courses on post-test SE and engagement (affective, behavioral, and cognitive), while controlling for their respective scores at pre-test. Nominal participant characteristics were analyzed using IBM SPSS Statistics, Version 27.0 (IBM Corporation, New York). All remaining analyses were performed using R studio version 1.3.959 (RStudio Team, Boston).

Results

Participant Characteristics

Descriptive statistics for student characteristics are summarized in Table 2. There were no significant differences between the course sections for gender identity (X²(3) = 5.949, p = 0.114). Significant differences were found between the two course sections for class standing (X²(6) = 42.625, p < 0.001, see Table 2). Post hoc testing revealed significant differences between the courses for percentage of freshmen (Z = 6.4), juniors (Z = 2.0), and seniors (Z = 2.2). There were no significant differences between the courses for percentage of high school students, sophomores, post-baccalaureate students, or students with unknown class standing.

Table 2.

Summary and Pearson’s chi-squared test of student demographic characteristics for the modified and standard courses.

	Modified Course (n = 277)	Standard Course (n = 217)	X2	P
Gender
Female	204 (73.6%)	179 (82.5%)	5.949	0.114
Male	64 (23.1%)	34 (15.7%)
Gender Non-conforming	1 (0.4%)	0 (0%)
Transgender Female	0 (0%)	0 (0%)
Transgender Male	0 (0%)	0 (0%)
Unknown	8 (2.9%)	4 (1.8%)
Class Standing
High School Student	5 (1.8%)	5 (2.3%)	42.625	0.000
Freshman	25 (9.0%)	69 (31.8%)*
Sophomore	108 (39.0%)	70 (32.3%)
Junior	57 (20.6%)	30 (13.8%)*
Senior	64 (23.1%)	33 (15.2%)*
Post-baccalaureate	7 (2.5%)	4 (1.8%)
Unknown	11 (4.0%)	6 (2.8%)

(%) is proportion within the course section.

*Adjusted Pearson residual >1.96.

Post hoc testing for student class standing between the course designs was done using adjusted Pearson residuals for each category. Significant differences between courses were found for freshman, junior, and senior class standing.

Student Participation

For the pronunciation assignment, students in both courses completed most weekly assignments, leading to a heavily skewed data set. There was a statistically significant difference in completion rate where students in the standard course had a higher rate of completion (M = .97, SD = .14) compared to the modified course (M = .94, SD = .13, X²(1) = 34.83, p < .001, η² = .07, 95%CI [.03, .11], see Table 3). However, the effect size indicates that the course type was a small factor. Completion rate for the metacognition assignment in the modified course was 88.0% (243/277). There was no equivalent assignment in the standard course for comparison.

Table 3.

Summary of results for student outcome measures for the modified and standard course.

		Course Delivery
		Modified Course		Standard Course		p valuesa
Student Outcome Measure		n	Mean (SD)	n	Mean (SD)
Student Participation
Pronunciation Assignment		254	.94 (.14)	196	.97 (.14)	<0.001
Student Performance
Average Exam Score		254	91.46 (4.12)	197	91.68 (4.31)	0.53
Course Grade		277	95.25 (4.05)	217	93.39 (4.23)	<0.001
Course Satisfaction
Student Satisfaction Questionnaire		153	4.24 (.49)	157	4.09 (.60)	0.02
Student Self-Efficacy (SE)
Medical Language SE Scale	Pre-Test	97	6.49 (1.56)	98	6.29 (1.48)	0.57
	Post-Test		7.73 (1.21)		7.55 (1.41)
p valuesb			<.001		<.001
Student Engagement
Student Engagement in School Scale
Affective Engagement	Pre-Test	93	4.16 (.43)	98	4.04 (.48)	0.82
	Post-Test		4.25 (.49)		4.25 (.49)
Behavioral Engagement	Pre-Test	97	3.73 (.47)	99	3.73 (.42)	0.06
	Post-Test		3.85 (.47)		3.75 (.47)
Cognitive Engagement	Pre-Test	98	3.80 (.57)	96	3.82 (.60)	0.13
	Post-Test		3.96 (.58)		3.87 (.59)

Abbreviations: SD, standard deviation; SE, self-efficacy.

The table includes mean and standard deviation for each outcome measure, with n indicating the number of students who completed the measure and were included in the analysis. For student self-efficacy and student engagement, only students who completed both the pre-test and post-test for that outcome.

^ap values indicate level of significance for statistical tests for between-group differences for the modified and standard courses for each outcome measure.

^bp values indicate level of significance for statistical tests for within-group statistical testing.

Student Performance

A significant difference was found between the standard (M = 92.1, SD = 7.9) and modified (M = 95.6, SD = 6.6) courses based on average final grade, t(476) = 4.87, p < .001, d = .45, 95% CI [.27, .63] (see Table 3). Students in the modified course earned, on average, higher final grades than students in the control course by nearly half a standard deviation. No significant differences between courses were found in average exam scores.

Student satisfaction

A significant difference in course satisfaction was found between the standard (M = 4.09, SD = .60) and modified courses (M = 4.24, SD = .49), t(308) = 2.40, p < .05, d = .27, 95% CI [.05, .5] (see Table 3). Students in the modified course were more satisfied with their experience than students in the standard course.

Student SE and Engagement

Students in both courses experienced significant gains in SE (standard, Z = 3849.5, p < .001; modified, Z = 4183.5, p < .001). No between-group differences were found for SE gains. After controlling for pre-test scores, we observed a marginally significant main effect for behavioral engagement, F (1,193) = 3.48, p =.06, η² = .01, 95% CI [.00, .05]. Students in the modified course tended to report greater behavioral engagement (M = 3.85, SD = .47) relative to students in the standard course (M = 3.75, SD = .50). No significant differences were found between the two courses affective engagement or cognitive engagement after controlling for pre-test scores (see Table 3).

Discussion

Without changing major elements of the overall course, including asynchronous course design, size, content, and assessments, a large, asynchronous online medical terminology course was modified using language learning and learning science principles. Course modifications provided opportunities for students to practice authentic language use, interact with their classmates and instructor, receive timely feedback on their language use, pace their learning, and increase their knowledge of metacognition. Students in both the modified and standard courses participated in most required course assignments and assessments and experienced growth in SE related to using medical terminology. Students in the modified course earned higher course grades, reported greater satisfaction, and may have experienced greater behavioral engagement when compared to the standard course.

Medical terminology is spoken, written, and listened to between professionals and between professionals and laypersons receiving care. Pronunciation assignments in the modified course were low stakes opportunities for students to practice authentic use of medical terminology. Medical terms were used in the context of a case description, and the assignment created speaker and audience roles. The audience of a small group of peers was consistent throughout the semester to build student comfort and avoid learner anxiety that may occur when practicing language with unfamiliar peers [5]. Creating a low-stakes environment to practice language use can reduce anxiety and improve performance in future use of the new language [5].

Small groups within large online courses can also promote collaborative learning among peers. For the modified course, peer feedback was leveraged to avoid time burdens for the instructor. Peer feedback promotes social interaction among classmates and can be used as a mechanism for social collaboration in learning in the online environment. To optimize the quality of peer formative feedback, students were trained in the use of a structured rubric with explicit definitions of different performance levels [8,27,28]. Each week, students were provided with structured feedback from multiple peers, which is superior to feedback from a single classmate [29]. Students were not required to interact with their small group outside of graded assignments, but they were encouraged to connect with peers as a means of group study, asking clarifying questions about course content or logistics, or troubleshooting technology issues. The extent that small groups are utilized by students to promote collaborative learning could be a focus of future research.

Class participation relates to academic achievement [30]. Student participation may be an indicator of task value, student judgement of interest, value, or importance of a task. Task value is an example of a motivational belief that is related to choice of learning activities and achievement [31–33]. The metacognition module was an optional assignment for the modified course only. Most students completed the module and were exposed to the content. Participation in the standard and modified versions of the pronunciation assignment was high, which may indicate that students found the assignment valuable to their learning. Participation in the modified pronunciation assignment was lower than the standard assignment. The modified assignment may have placed a greater demand on students when compared with the standard version. The additional effort with no additional point value for the assignment (i.e., total points) may have contributed to the difference in student participation. Importantly, the modified assignment was accessible to all students through the learning management system used to administer the course. The standard assignment required the purchase of software which functioned best with use of a smartphone, which can pose potential access barriers for some students.

Our results show that overall course grade in the modified course was significantly higher than in the standard course. This difference is likely explained by student performance on weekly summative assessments because there were no between-course differences found for summative exam scores. Despite the differences found, students in both courses showed high levels of performance as indicated by their summative exam scores and overall course grades. Because this medical terminology course is a prerequisite for many programs, students in both sections of this course likely related this material very closely to future professional goals, which may be associated with intrinsic motivational behaviors and achievement [33].

Achievement emotions are receiving more attention from medical educators, yet there remains a paucity of research investigating the relationship between course design elements and achievement emotions. Achievement emotions may impact academic outcomes [31]. Positive emotions, such as satisfaction, may have important effects on long term achievement outcomes, and negative emotions may impact course-related achievement outcomes [6,31]. Achievement emotions may relate to students’ use of learning strategies, choice of future courses, and academic achievement [11,34,35]. Our results show that students in the modified course reported greater satisfaction when compared with the standard course. This may have implications to longer term achievement outcomes for the students in the modified course, but this is beyond the scope of the present study.

Our results showed growth in SE for both courses with no between-course differences. Previous evidence indicates a positive relationship between SE and academic outcomes, which may be in part due to achievement behaviors of students with strong SE. These students may use more effective learning strategies and be willing to put forth greater effort to overcome setbacks when compared with students with a weaker sense of SE, thus achieving better academic outcomes [13,36,37]. Others have shown significant and positive relationships between student SE, course enjoyment, and performance [31]. Established measures of student SE for learning medical language are limited. Previous research used tools designed for learning English or adapted versions of other measures [7,38]. In the present study, our measure of SE was adapted from a published SE measure [25]. There is currently no evidence for psychometric properties of the adapted version of this measure. Future research and outcome measures with strong psychometric properties is needed to further understand SE as a student outcome in undergraduate asynchronous courses.

Student engagement is multi-dimensional and includes behavioral, affective, and cognitive domains [26]. An asynchronous online course, by nature, offers fewer organic opportunities for students to interact with the instructor or peers, which may lead to learner isolation, reduce overall student engagement, and negatively impact student achievement [39]. Current evidence is conflicting about the optimal target for strategies attempting to increase student engagement, whether it is peer-peer, student-instructor, or student-content interaction. While some have reported that peer-peer and student-content is more important for positive student outcomes [19], others have reported that student-instructor interaction is most important [20,21]. Course modifications in the current study aimed to impact on student through increased student-student interaction and student-instructor interaction. Our results showed marginal differences in behavioral engagement among students in the modified and standard courses, and no between-course differences were found for affective and cognitive engagement. Importantly, items on the SESS are not specific to one course but ask more generally about student engagement in an academic setting. It may be that student engagement within an isolated course may vary from their overall engagement as measured by the SESS.

Conclusions

Modifications were made to the learning environment for an asynchronous online medical terminology of undergraduates. Students showed superior performance, higher course satisfaction, strong participation, and greater behavioral engagement when compared with the standard course design. Summative exam score, affective engagement or cognitive engagement were not different between groups. Small changes to course delivery made significant differences to students.

Limitations

Differences in student standing between the modified and standard courses may have affected the outcomes measured. This study focused only on an entirely asynchronous undergraduate online course in medical language. Therefore, generalizability of these findings may be limited to similar course offerings. Multiple modifications were made to the course and compared to the standard course, so the effects of individual modifications cannot be determined from this study. Long-term retention of knowledge, effects on individual students, and future medical language use were not assessed and cannot be determined from the present study. Lastly, analyses comparing pre- and post-test scores, including the Medical Language SE scale and SESS, were limited to students who completed these measures at both time points. These measures were not course requirements, and this may have impacted the number of students completing both, leading to lower student participation than expected.

Funding Statement

This research was partially funded by the National Center for Advancing Translational Sciences [Scott, TL1TR002735].

Disclosure statement

No potential conflict of interest was reported by the author(s).

Authors’ contributions

Conception and design of the study: KS, JY, MNN. Data collection: KS. Statistical analysis: JB. Discussion and revision of the manuscript: KS, JY, JB, MNN. All authors have read and approved the final version of the manuscript.

Availability of data and material

The datasets generated and analyzed during the current study are available from the corresponding author upon request.