Abstract
Background
The concepts of online and blended education came into the limelight in the 19th century. Over time, the concepts expanded and reached a peak in 2021 in response to the COVID-19 lockdown. One of the challenges is the monitoring of the performance of distant learners. In face-to-face courses, an instructor can easily identify struggling learners during the regular meetings.
Aim of the Study
This study explored variables that can predict the academic achievement of learners early in online learning environments. Although there was no consensus, the factors were still hypothesized as predictors for academic achievement.
Methods
A quasi-experimental study was conducted to test the hypothesis. Thirty-three graduate learners were enrolled in a blended trauma course. The learners’ age, their previous experiences in online education, pre-test scores, and the number of logs to the online platform were studied. These elements were considered as predictors of academic achievement in the online aspect of the course.
Results
The findings revealed that there was no statistically significant correlation between the age, the previous experience in online education, the pre-test scores, and the number of logs in the first two weeks. However, there was a statistically significant correlation between the number of logs into the online platform in the first three weeks of study and the learners’ academic achievement. Additionally, the number of logs in the first three weeks was a statistically significant predictor for academic achievement in online education. This early prediction can help instructors to identify and support struggling learners.
Conclusion
The records of the online activity of learners in the first three weeks of study can help in early prediction of their academic achievement. Age, previous online education, and pretest scores were not statistically significant predictors.
Keywords: academic achievement, prediction, learning analytics, online education, graduate learning
Introduction
The modern concept of distance and hybrid “blended” learning started with the progress of postal services in the 19th century. The first offer of distance degrees was in the University of London, in 1858.1
Distance education is an education that emphasizes using of technology for learning purposes, without being physically present in a traditional classroom. It is the creation of learning access when the teacher and the learners are detached by time and distance.2
Hybrid learning is an approach that combines face-to-face with internet-mediated instructions. Hybrid learning is also called blended learning, and mixed-mode learning. Hybrid learning has the advantages of flexibility in time, pace, and place to learn. Moreover, hybrid learning keeps the advantages of face-to-face learning which is still important in medicine to achieve the proposed clinical competency.2
In 2012, the concept of Massively Open Online Courses (MOOCs) was launched. Large learning management systems such as Blackboard or MOODLE have been used for online learning purposes. Two categories of learning management systems are described: commercial learning management systems and open-source learning management systems. The commercial LMS are paid platforms like Blackboard, SchoolGen, and WebCT. However, the non-commercial LMS are free open-source software as MOODLE, Claroline, and Bazaar.3
Higher education institutions have used Virtual Learning Environments (VLEs). A VLE can be defined as a group of software tools that reinforce administrative activities, learning, and scientific research utilizing the World Wide Web. Various recognized VLEs use built-in social media formats within their design.4
Regarding the Oxford University computing services, web learning can be divided into 5 main pillars: Information for news sharing, notifications, and other information such as policies and rules, programs and courses specifications as well as schedules; Content for providing various resources like lecture notes, e-books, educational videos and games, online simulations, and hyperlinks to external online sources; Communication that can be achieved through various tools such as e-mail, discussion forum, messaging applications, and wikis; formative and summative assessments that can be performed in various forms as quizzes and assignments; Management tools that can maintain planning and delivery of online courses as registration for courses, monitoring of enrolled students, and the processing of students’ grades.5
A modular object-oriented dynamic learning environment (MOODLE) is the officially adopted online learning management system in our institute. MOODLE is an open-source LMS. The instructors of the online courses can monitor the learners’ activities on the LMS.6
The virtual learning environment and the learning management systems can offer a central focus for both students and staff, wherever their geographic location. This allows the flexibility of time giving the two options of synchronous and asynchronous learning.4,7
Predicting learners’ academic achievement in online education is challenging. However, it warns educators for the less engaged and struggling learners who are liable to fail or get low scores. Various variables may predict the achievement of learners in the online environment as the age of learners, their previous participation in online learning, marital status, previous academic achievement, and online activities during the course to measure the learners’ engagement. Learning analytics is collecting and analyzing data about the performance of learners in the online environment, for improving learning through detecting learners at-risk of low academic achievement. Online predictors can be the number of times the learner logged into the course, the number of activities that the learner completed on the platform, and the time spent on the platform.8
As continuous monitoring is an important aspect of effective online teaching, it was recommended in the literature to carry out more studies to explore the predicting factors either personal or educational variables. Studying the variables that can predict the academic achievement of learners in distance education can improve the achievement of learners. This goes with the concept of “prevention is better than cure”. It can help the online instructors design corrective educational strategies for the students who are expected to get poor academic performance. As there is no consensus regarding the early predicting factors of the academic achievement of learners in online education, our focus in this research is to explore the possibility of the prediction of the academic achievement of learners early in the hybrid course.9–11
Methodology
Predictive correlation design was used in this research. Thirty-three postgraduate learners were enrolled in a hybrid trauma b course. The minimum sample size needed was calculated using the “G- power” software for calculating the sample size. Using a study power of 90%, type I error of 1%, and effect size of 0.85. Online learning was in the form of synchronous and asynchronous sessions. The online aspect was created on Moodle. The face-to-face aspect was in the form of clinical ward rounds, clinics, and operative theater.
The course was organized into themed weeks. Week one for orientation to the course, the online platforms, and the applications to be used in the online aspect of the course. In week 2 was the biomechanics of fractures. Week 3 exposed the learners to pathological fractures. Week 4 discussed the periprosthetic fractures. Week 5 was the time for osteoporotic fractures. Childhood injuries were discussed in week 6. The theme of week 7 was polytrauma patients. In week 8, joints dislocation was introduced. Week 9 was the time for discussing implant failure in trauma cases. In week 10, the compartmental syndrome was the learning theme. During week 11, learners learned about injuries in elderly patients. And lastly, sports injuries were introduced in week 12.
The online contents were presented as small chunks with descriptive headings and subheadings, to avoid cognitive load, respecting the attention span of learners. The arrangement of the course materials offered a simple format and guaranteed easy navigation. Moreover, each section (weekly contents) was on one screenshot (no need to scroll down the page), not to distract the learners (Figure 1). The e-learning materials including a schedule, course objectives, assessment plan, lecture notes, books, links for e-books, links for videos, links for gaming, assigned tasks, electronic discussion groups and quizzes were uploaded to the Moodle. They were organized into sections aligned with the sequence of the course.
Figure 1.
Trauma B online platform on MOODLE.
A detailed plan for each topic was created to fulfill the learning outcomes. Touch surgery application was used for gaming. Touch surgery offered quizzes with immediate feedback for the learners. In addition, Vumedi website was used to offer videos of detailed surgical approaches, procedures, tips, and tricks.
As regards the assignments, instructions on how to complete the assignments were created and uploaded online. Due dates for submitting the online assignments were clear. Feedback and grading of the assignments were delivered regularly. These online learning activities exposed the learners to various experiences as answering script concordance test,12 creating scripts in script-based learning activities,13 playing a game then taking a test, quizzes, and discussions. Feedback was delivered by the instructor regularly. The format and the maximum size of the assignment files were specified to the learners for each activity (500 MB).
WhatsApp group was created. Through this group, notifications, reminders, and links for the synchronous online meetings were sent. Learners were allowed to post on the group. They were welcomed to send their inquiries and suggestions.
Online synchronous sessions were delivered using zoom platform. An orientation session was delivered online. The netiquette was clarified to the learners. The link for the zoom meetings was sent to the learners via WhatsApp, and it was launched on Moodle.
Each online synchronous session was 1–2 hours long. Sessions were recorded and shared with learners after the session. The online whiteboard of each meeting, where ideas were recorded, was saved, and shared with the learners. Moreover, the chat was saved and shared. Sessions included discussion of approaching, diagnosis and management of clinical cases. Learners were asked to complete the recommended readings and watch the recommended videos before the synchronous meeting. After the session, learners worked on the assigned task.14
For example, the “Pathological fractures” session:
A clinical problem about pathological fracture was presented.
Learners had to read the assigned materials first.
Learners were asked to read the assigned problem.
Learners were asked to create a script for the case. In the script, the learner had to decide which of the provided information helped him reach the diagnosis. What was the surgical and/or non-surgical treatment appropriate for the case, instruments needed, relevant anatomy, key surgical steps, key points of assistant performance, post-operative care, and postoperative information supplements?
Learners received feedback from the instructor.
Synchronous discussion on zoom was offered to resolve any misunderstanding.
Regarding the asynchronous discussions, a discussion forum was used on Moodle. Probing question or statement was provided, and the learners were invited to discuss. Netiquette for using the discussion was clarified.
Age of learners and their previous experience in online courses were collected using online short form that is nominated (to be able to correlate data of learners to their grades). We used the number of logs into the online platform of the course (Moodle) in the first three weeks (learning analytics). The online pre-test that was used to explore the level of knowledge before taking the course was also considered as a predicting variable. Moreover, learners’ grades in the written exams were used in the analysis as the academic achievement indicator. The written exams were in the form of continuous assignments and a final test. The continuous assignments were intermediate-stake assessment points. Marks of the assignments accounted for 40% of the total written-assessment marks. However, the marks of the final written exam were 60% of the total written-assessment marks. The final exam contained 50% MCQs and 50% modified essay questions.
This research work aimed at answering the following question:
What are the predictors of learners’ academic achievement in online environment?
Quantitative data were coded, checked for errors, and entered Statistical Package for the Social Sciences (SPSS) Version 22.0 Software for statistical analysis. Shapiro–Wilk test was used to evaluate normality of distribution of data. Spearman and Pearson's correlation were used to study the association between variables. Regression analysis was used to formulate the prediction model.
Ethical Considerations
Study procedures were reviewed and approved by the Alexandria Faculty of Medicine research ethics committee.
Informed consent was obtained from the study participants prior to study commencement.
Results
Shapiro–Wilk test of normality showed that grades of written exams (W1), pre-test scores (W2), and number of logs (W3) were normally distributed (Normally distributed data by Shapiro–Wilk test (W1 = 0.97, p1 = 0.37), (W2 = 0.97, p2 = 0.44), (W3 = 0.96, p1 = 0.65)). However, age and number of previous online courses were not normally distributed. Statistical significance was set at the level of p ≤ 0.05 in all the statistical tests.
No statistically significant correlation was found between age of learners and the academic achievement [Spearman correlation coefficient (r) = −0.61, p = 0.74].
There was no statistically significant correlation between number of previous online courses and the academic achievement [Spearman correlation coefficient (r) = −0.042, p = 0.82]. Additionally, the correlation between pre-test scores and the academic achievement by the end of the course was not significant [Pearson’s correlation coefficient (r) = −0.89, p = 0.81].
There was not statistically significant correlation between the number of online logs in the first week of the course and the academic achievement [Pearson’s correlation coefficient (r) = 0.61, p = 0.58]. Similarly, correlation between the first 2 weeks of study and the academic achievement was statistically insignificant. [Pearson’s correlation coefficient (r) = 0.53, p = 0.52]. There was a significant correlation between the number of online logs in the first three weeks of the course and the academic achievement [Pearson’s correlation coefficient (r) = 0.89, p ˂ 0.01].
As there was no statistically significant correlation between age, number of previous online courses, pre-test scores, number of logs in the first week and number of logs in the first 2 weeks, and the academic achievement of learners. These variables were excluded from our assumptions as being predictors for the academic achievement. They were not included in the regression model. The number of logs in the first 3 weeks was the variable under study as a predictor for the academic achievement in the linear regression model.
On examining the residuals of the distribution of the number of the three-weeks logs, the scatter plot showed heteroscedasticity (Figure 2). Heteroscedasticity was fixed using weighted regression. Moreover, the Durbin-Watson statistic was 2.17 (between 1.5 and 2.5) and therefore the data were not autocorrelated.
Figure 2.
Residual plot for assessing homoscedasticity of data.
On testing the other assumptions of linear regression, Cook’s values for all learners were below the cutoff point (4/n = 0.12). This means that there were no influential outliers in the predictor variable (count of logs) (Figure 3).
Figure 3.
Cook’s distance for individual learners.
Weighted linear regression was used to test if count of logs into the online platform in the first 3 weeks of the course significantly predicted the academic achievement of learners (scores in written exams).
The fitted regression model was academic achievement (exam scores) = 3.48 + 0.75*(count of logs into the online platform in the first 3 weeks of the course).
The overall regression was statistically significant (R2 = 0.89, F (1, 31) = 257.86, p = ˂0.00).
It was found that the count of logs into the online platform in the first 3 weeks of the course significantly predicted the academic achievement of learners (β = 0.75, p = ˂0.00) (Table 1).
Table 1.
Regression Model Between Count of Three-Weeks Log into the Online Platform and the Academic Achievement of Learners
| Modela,b | Unstandardized Coefficients | Standardized Coefficients | t | Sig. | |
|---|---|---|---|---|---|
| B | Std. Error | Beta | |||
| (Constant) | 3.477 | 4.637 | 0.750 | 0.459 | |
| Three weeks logs | 0.747 | 0.047 | 0.945 | 16.058 | 0.000* |
Notes: aDependent Variable: academic achievement. bWeighted Least Squares Regression - Weighted by weight. *Significance at p ≤ 0.05.
Discussion
Prediction of academic achievement in online environment can help the instructors to detect learners liable to get low scores. Our hypothesis was the creation of a prediction model for the academic achievement in online education through studying the age of learners, number of previous online courses, pre-test scores and number of logs into the online platform of the course. The age of learners was excluded from the model as it was not correlated with the learners’ grades. This finding was consistent with that of Amro et al.15 Similarly, Kovačić16 reported that age does not have a significant impact on predicting academic success. This can be explained by the impact of confounders that affect this relationship. Confounders as ability to deal with technology, willingness to learn new skills, time spent studying, social duties and responsibilities and previous experience in online learning. With larger sample, there could be a significant correlation between age and academic achievement. However, the meta-analysis study of Sturman17 revealed that age was correlated with the academic performance without studying the impact of age on the academic achievement. Additionally, Park et al18 revealed that the age of the students significantly predicted their academic achievement. They revealed that those with younger age who are familiar with the technology performed better in the online courses. Regarding the association between previous online courses and the academic achievement of learners in our study, there was no significant correlation between the number of previous online courses and the academic achievement in the hybrid trauma course. This finding is consistent with that of Nagel19 who reported that there was no statistically significant impact of previous experience of online courses on the academic achievement of learners. This finding may be explained by that, academic achievement is affected by the design of the course, the ability and preferences of the learner rather than his previous experience.2
As regards the previous experience in online learning, Regarding the association between previous online courses and the academic achievement of learners in our study, there was no significant correlation between the number of previous online courses and the academic achievement in the hybrid trauma course. This finding is consistent with that of Nagel18 who reported that there was no statistically significant impact of previous experience of online courses on the academic achievement of learners. This finding may be explained by that, academic achievement is affected by the design of the course, the ability and preferences of the learner rather than his previous experience.20
On studying the association between the pre-test scores and the academic achievement (post-test scores), there was no statistically significant correlation. This finding is consistent with that of Mitra et al.21 This can be explained by the uniformly higher range of posttest score that would not correlate well with the pre-test score. Due to insignificant correlation, the previously mentioned variables were excluded from the prediction regression model.
The correlation between number of logs into the online platform of the course and the academic achievement became statistically significant after the third week of the course. Additionally, count of logs into the online platform in the first 3 weeks of the course significantly predicted the academic achievement of learners. Guidry K.10 series showed consistency with our results as regards the correlation between the online activity and the academic performance. Moreover, Rajabalee et al22 revealed consistent results as regards the association between the online activities and the academic achievement. This can be explained by the internal motivation of students to learn. The more motivated the learner, the more login activities to the platform of the course.
Limitations
The sample size was relatively small. However, the whole population (learners enrolled in the online Trauma b course were involved). For this sample size (33 participants) only 3 predicting variables could be considered understudy but only one variable was eligible for inclusion. Then the small sample size was considered.
Conclusion
This research work concludes that the records of the online activity of learners in the first three weeks of study can help in early prediction of their academic achievement. Age, previous online education, and pretest scores were not statistically significant predictors. Monitoring the online activities of learners on the educational platforms can enable the instructors to pick the struggling learners. This could help improving the educational impact and the outcomes of the online education. Future research with larger sample size is recommended. Additionally, studying the time spent on the online platform of the course as a predicting factor for the academic achievement of learners could be an interesting point of research that adds to the literature.
Acknowledgments
We appreciate the support of the Orthopedic and traumatology department for their support. We deliver deep thanks for the postgraduate students who agreed to participate in this study.
Data Sharing Statement
Data are available on request.
Author Contributions
All authors made a significant contribution to the work reported, whether that is in the conception, study design, execution, acquisition of data, analysis and interpretation, or in all these areas; took part in drafting, revising or critically reviewing the article; gave final approval of the version to be published; have agreed on the journal to which the article has been submitted; and agree to be accountable for all aspects of the work.
Disclosure
The authors declare that they have no conflict of interest.
References
- 1.Kent M, Leaver T. Chapter 1: the revolution that’s already happening. In: An Education in Facebook? Higher Education and the World’s Largest Social Network. New York: Taylor & Francis 2014; 2014:1–10. [Google Scholar]
- 2.O’byrne W, Pytash K. Hybrid and blended learning. J Adolesc Adult Lit. 2015;59:137–140. doi: 10.1002/jaal.463 [DOI] [Google Scholar]
- 3.Alqahtani A. The Effectiveness of Using E-learning, Blended Learning and Traditional Learning on Students’ Achievement and Attitudes in a Course on Islamic Culture: an Experimental Study [PhD diss.]. University of Durham; 2010. [Google Scholar]
- 4.Singh V, Thurman A. How many ways can we define online learning? A systematic literature review of definitions of online learning (1988–2018). Am J Distance Educ. 2019;33(4):289–306. doi: 10.1080/08923647.2019.1663082 [DOI] [Google Scholar]
- 5.Mayer RE. Thirty years of research on online learning. Appl Cogn Psychol. 2019;33(2):152–159. doi: 10.1002/acp.3482 [DOI] [Google Scholar]
- 6.Paudel P. Online education: benefits, challenges and strategies during and after COVID-19 in higher education. Int J Stud Educ. 2021;3(2):70–85. doi: 10.46328/ijonse.32 [DOI] [Google Scholar]
- 7.Mishra L, Gupta T, Shree A. Online teaching-learning in higher education during lockdown period of COVID-19 pandemic. Int J Educ Res Open. 2020;1:100012. doi: 10.1016/j.ijedro.2020.100012 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Baashar Y, Alkawsi G, Mustafa A, et al. Toward predicting student’s academic performance using Artificial Neural Networks (ANNs). Appl Sci. 2022;12(3). doi: 10.3390/app12031289 [DOI] [Google Scholar]
- 9.Amin IM, Hassan NC, Jalil HA. Predictors of academic achievement in online peer learning among undergraduate students in a Malaysian public university. Int J Acad Res Bus Soc Sci. 2018;7(14):564–587. [Google Scholar]
- 10.Guidry K. Predictors of student success in online courses: quantitative versus qualitative subject matter. J Instr Pedagog. 2013;10(April):1–11. [Google Scholar]
- 11.Namoun A, Alshanqiti A. Predicting student performance using data mining and learning analytics techniques: a systematic literature review. Appl Sci. 2020;11(1):237. doi: 10.3390/app11010237 [DOI] [Google Scholar]
- 12.Gawad N, Wood TJ, Cowley L, Raiche I. The cognitive process of test takers when using the script concordance test rating scale. Med Educ. 2020;54(4):337–347. doi: 10.1111/medu.14056 [DOI] [PubMed] [Google Scholar]
- 13.Zhao X, Cong L. Nurse education today effect of problem and scripting-based learning combining wearable technology on orthopedic operating room nurses’ learning outcomes. Nurse Educ Today. 2019;73:13–16. doi: 10.1016/j.nedt.2018.11.005 [DOI] [PubMed] [Google Scholar]
- 14.Kurt S. Flipped classroom. Educational Technology; 2018. Available from: https://educationaltechnology.net/flipped-classroom/. Accessed March 2022. [Google Scholar]
- 15.Amro HJ, Mundy M-A, Kupczynski L. The effects of age and gender on student achievement in face-to-face and online college algebra classes. Res High Educ J. 2015;27(January):1–22. [Google Scholar]
- 16.Kovačić ZJ. Early prediction of student success: mining students enrolment data. In: Proceedings of informing science and IT education conference; 2010:647–665. [Google Scholar]
- 17.Sturman MC. Searching for the inverted U-shaped relationship between time and performance: meta-analyses of the experience/performance, tenure/ performance, and age/performance relationships. J Manage. 2003;29(5):609–640. [Google Scholar]
- 18.Park K, Moon S, Oh J. Predictors of academic achievement in distance learning for nursing students. Nurse Educ Today. 2022;108:105–115. doi: 10.1016/j.nedt.2021.105162 [DOI] [PubMed] [Google Scholar]
- 19.Nagel T. Academic Achievement and Persistence in Online Self-Paced Courses [PhD diss]. University of Missouri-Columbia; 2016. [Google Scholar]
- 20.Abuhassna H, Al-Rahmi WM, Yahya N, Zakaria MAZM, Kosnin ABM, Darwish M. Development of a new model on utilizing online learning platforms to improve students’ academic achievements and satisfaction. Int J Educ Technol High Educ. 2020;17:1–23. doi: 10.1186/s41239-020-00216-z [DOI] [Google Scholar]
- 21.Mitra NK, Aung HH, Kumari M, et al. Improving the learning process in anatomy practical sessions of chiropractic program using e-learning tool. Transl Res Anat. 2021;23:100100. doi: 10.1016/j.tria.2020.100100 [DOI] [Google Scholar]
- 22.Rajabalee BY, Santally MI, Rennie F. A study of the relationship between students’ engagement and their academic performances in an eLearning environment. E-Learn Digit Media. 2020;17(1):1–20. doi: 10.1177/2042753019882567 [DOI] [Google Scholar]