Abstract
Background
Endoscopic examination is a popular and routine procedure for the diagnosis and treatment of gastrointestinal (GI) diseases. Skilled endoscopists are in great demand in clinical practice, but the training process for beginners to become endoscopy specialists is fairly long. Convenience and a self-paced, learner-centered approach make electronic learning (e-learning) an excellent instructional prospect.
Objective
This study aimed to develop and apply an e-learning system in gastroscopy teaching and learning and to evaluate its effectiveness and user satisfaction.
Methods
The e-learning software Gastroscope Roaming System was developed for primary learners. The system simulates the real structure of the upper gastrointestinal (UGI) tract to teach the main characteristics of gastroscopy under both normal conditions and conditions of common UGI tract diseases. A randomized controlled trial was conducted. Participants were randomly allocated to an e-learning group (EG)or a non–e-learning control group after a pretest. On completing the training, participants undertook a posttest and gastroscopy examination. In addition, the EG completed a satisfaction questionnaire.
Results
Of the 44 volunteers, 41 (93%) completed the gastroscopy learning and testing components. No significant pretest differences were found between the intervention and control groups (mean 50.86, SD 6.12 vs mean 50.76, SD 6.88; P=.96). After 1 month of learning, the EG’s posttest scores were higher (mean 83.70, SD 5.99 vs mean 78.76, SD 7.58; P=.03) and improved more (P=.01) than those of the control group, with better performance in the gastroscopy examination (mean 91.05, SD 4.58 vs mean 84.38, SD 5.19; P<.001). Overall, 85% (17/20) of the participants were satisfied with the e-learning system, and 95% (19/20) of the participants considered it successful.
Conclusions
E-learning is an effective educational strategy for primary learners to acquire skills in gastroscopy examination and endoscopic imaging of the GI tract.
Trial Registration
Chinese Clinical Trial Registry ChiCTR-IOR-17013091; http://www.chictr.org.cn/showproj.aspx?proj=22142
Keywords: e-learning, gastroscopy, endoscopy, randomized controlled trial
Introduction
Background
With the development of digestive endoscopy technology, gastrointestinal (GI) diseases are increasingly treatable by digestive endoscopy. Skilled and experienced endoscopists are needed in clinical practice worldwide, yet the training process for primary learners is difficult and time consuming. Hence, educating endoscopists has become a global challenge in medical education.
Traditionally, experienced endoscopists have played an important role in training primary learners, and primary learners have worked with their tutors to practice endoscopy on patients [1]. It is common for primary learners to feel nervous and anxious and to be unable to obtain satisfactory cooperation from their patients. In addition, patients may refuse to permit primary learners to practice GI endoscopy on them. Traditional clinical training may fail to provide adequate information on multiple needs and cognitive deficits. Endoscopic simulators, including ex vivo animal tissue models, live animal models, mechanical models, and virtual reality computer simulators [2,3], have been widely used in academic practice. However, problems remain with regard to the high cost of teaching, limited resource space, and small audience, and primary learners can only study the structure of the upper gastrointestinal (UGI) tract and cannot further study the endoscopic manifestations of UGI diseases. Thus, improving the instructional programs and models for primary learners is a key issue in endoscopy training.
Many institutes are pioneering electronic learning (e-learning) methods as a cost-effective alternative to traditional methods [4,5]. E-learning is a well-established approach to learning via electronic-/computer-based, mostly Web-based, programs [6]. The technology is based on the use of the internet to deliver a broad array of educational materials and training procedures to enhance knowledge and performance [7,8]. Numerous researchers have described the use of e-learning in medical education [9-18]. Compared with traditional teaching environments, e-learning has several advantages, such as asynchrony, cost savings, individualized learning, greater accessibility, greater ease of distribution, and up-to-date content, which may overcome the difficulties and dilemmas experienced during the early phases of endoscopic training [19-22]. Learners may be attracted to e-learning because it centers them in the learning procedure, in contrast to their role as passive recipients in traditional training methods [23]. Learners enrolled in an e-learning program can choose the content that they access, the sequence in which it is studied, and the space used for learning based on their individual experience and personal learning objectives without time or space limitations [24,25].
Aim of This Study
In this context, we designed and developed the Gastroscope Roaming System (GRS) to create, assess, and implement an integrated e-learning gastroscopic education package for primary learners. The system focuses on teaching the principal characteristics of gastroscopy examination of the human UGI under normal conditions and under conditions of routine UGI diseases. We aimed to (1) develop a gastroscopy e-learning system for primary learners, (2) evaluate the efficacy and effectiveness of the system, and (3) assess the satisfaction of primary learners.
Methods
Development and Description of the Electronic Learning System
We collaborated with the Harbin University of Science and Technology to develop the GRS. The system was built to be bilingual in Chinese and English; the language can be freely switched on the log-in interface, and the system can be used by primary learners of gastroscopy in China and worldwide. First, the user interface of the system was designed and created. A 3-dimensional model of the UGI tract, including the esophagus, stomach, and duodenum, was obtained after a review of various data sources (mainly pictures and videos of gastroscopy examinations from the Digestive Endoscopy Center of the Second Affiliated Hospital of Harbin Medical University [HMU]) and was modeled by Unity 3D, Autodesk Maya, Zbrush, and Substance Painter software. Then, the 3-dimensional model was transformed into an operation scene, and the physiological function was simulated and rendered (Adobe Photoshop software for material design and production and Maya software for animation). The C++ programming language was used to design the roaming, interactive, and cognitive functions of the digestive organ structure and to realize the camera placement, path switching, and perspective switching. In addition, embedded links and demonstrations of lesion cases were created. Currently, the program requires a specified account number and password for a user to log in.
The two GRS modules were designed for primary learners to gain knowledge of the UGI anatomic structure, relevant endoscopy images, and protocol requirements and to improve their confidence. Module A contains normal human anatomic structures, with anatomic color illustrations from the pharyngeal portion to the descending part of the duodenum of the human UGI. Module B contains UGI images showing both normal conditions and conditions of routine UGI diseases. The software can also simulate UGI endoscopy and the route of the gastroscopy lens. The incidences of typical diseases along the endoscopic examination path are labeled based on clinical images of these diseases. Dozens of common UGI diseases are included for learning purposes. There are corresponding bilingual explanations under the images, including disease characteristics and endoscopic manifestations. Screenshots are presented in Figure 1.
Figure 1.
Gastroscope Roaming System’s software screen captures. (A) Software system log-in interface. (B) Normal esophageal entrance screenshot. (C) Normal cardia entrance screenshot. (D) Normal pyloric entrance screenshot. (E) Normal duodenal papilla screenshot. (F-I) Screenshots of common diseases in different parts of the disease learning state, and the disease description and gastroscopic characteristics are given below each image.
Participant Recruitment and Implementation Strategy
We issued an experimental information notice to recruit participants in the hospital. The 44 volunteers were all master degree students majoring in gastroenterology and hepatology at the Second Affiliated Hospital of HMU who had signed a consent form to participate in this trial. The teachers at the endoscopy center verified the basic information of each volunteer and played a key supervisory role in the entire process of learning and evaluation. All participants had acquired basic GI theoretical knowledge in their previous years of study, but because the GRS was designed for primary learners, the participants could not perform a gastroscopy and confirmed that they had not yet received systematic professional education in gastroscopy. In addition, they needed access to a procedure on their own computers to operate the sample version of the e-learning system. All participants completed a pretest questionnaire; those who did not complete the pretest or did too well in the pretest (correct rating >90% or higher) were excluded because the e-learning was designed for beginners who had not previously mastered sufficient endoscopy knowledge [26].
The experiment was conducted from November 2018 to December 2018. The flow of the experiment is shown in Figure 2. The 44 eligible volunteers were asked to complete the pretest in 30 min, and the unqualified participants were eliminated based on the results. The randomization sequence was computer generated using IBM SPSS version 24. Then, the qualified participants were randomly divided into two groups: the e-learning group (EG) and the non–e-learning control group (CG). The enrolled students spent 1 month in theoretical and practical training for gastroscopy examinations in our on-campus endoscopy center (Endoscopy Center, the Second Affiliated Hospital of HMU). A specific username and password were assigned to each EG participant to log in to the GRS during the trial. The EG had access to traditional learning combined with e-learning at any time and location. During the same period, the CG did not have access to the e-learning tool and learned only through the traditional mode of reading textbooks and other written materials in addition to practical training. All participants received routine gastroscopy training at the endoscopy center during the trial.
Figure 2.
Flow diagram showing the details of the participant enrollment, randomization, and tests. e-learning: electronic learning.
After the 1-month gastroscopy study, all participants were required to complete the posttest and gastroscopy examination assessment. The EG also completed an additional questionnaire to determine the level of satisfaction with the GRS.
Learning Outcome Evaluation Strategy
The potential and advantages of e-learning may not always be perceived as leading to significant improvements in educational outcomes. Therefore, it is necessary to demonstrate the rationality of e-learning through the assessments of effectiveness and efficacy. Kirkpatrick built a famous framework to evaluate the learning effect in the 1950s [27], which can be used to assess GRS effectiveness [24,28,29]. The framework describes four assessment levels: reaction, learning, behavior, and results [19,30]. Most studies that evaluate e-learning processes rely only on users’ learning and reaction rather than behavior and results [21,30-33]. GRS was designed to help improve the gastroscopic practice skills of primary learners. On this basis, e-learning in this study was further evaluated at the behavior level to reveal the efficiency and reliability of e-learning in medical education.
Pre- and Postintervention Tests
For the learning level assessment, the most conventional and reasonable test method is the comparison of pre- and postintervention test scores. In our experiment, the participants completed a pretest before gastroscopy learning and a posttest after 1 month of training. The pretest and posttest contained the same set of questions, but the order of the questions differed to ensure that the difficulty and reference standards of the two tests were at the same level [23,34]. The tests consisted of 50 multiple-choice questions, each of which had 4 to 5 response options and a single correct answer. Each question had a value of two points, with a total possible score of 100 points. The main topics of the multiple-choice questions were the elements of the basic operation of gastroscopy, cognition of the anatomical structure of the UGI, mastery of the characteristics of the structure of the UGI under gastroscopy, and the identification of and distinction between images of various common diseases of the UGI under gastroscopy. Most endoscopic images used in the questions were taken from the linked pictures used in the GRS system, and the rest were from our endoscopy center. The test was compiled by the authors of this paper and the designer of the GRS software. In total, 2 endoscopy experts from the Department of Gastroenterology, the Second Affiliated Hospital of HMU, reviewed the test questions to ensure that they were suitable for this experiment. The test paper (pretest version) is provided as Multimedia Appendix 1.
To ensure the fairness of the test and the reliability of the scores, we placed the participants in a classroom with a signal shield and conducted a closed-volume test with 2 invigilators and timers. The examination time was set at 30 min. When the time ended, the participants were asked to stop, and the papers were collected immediately. After the pretest, the participants could not see their own answers or the correct answers to ensure the validity of the posttest.
Gastroscopy Examination Test
As our experiment was based on the combination of e-learning and traditional endoscopy teaching (ie, blended learning), we tested the participants’ actual gastroscopy examination performance to evaluate the behavior level. After 1 month of training, the students undertook a gastroscopy examination, with the examination order based on drawing lots, and made real-time video recordings. All the patients participating in the gastroscopy test had taken an appointment for a general gastroscopy at the outpatient department and had agreed to a gastroscopy performed by primary endoscopists. As many unexpected problems appeared in the actual operation, such as patients with a special physiological structure or patients with lesions who were difficult to identify, teachers were present during the procedure. Each participant had three opportunities to select the performance that he or she thought was the most satisfactory for archiving. A total of 5 expert endoscopists (from the Affiliated Hospital of HMU) who were not aware of the participants’ grouping scored the students’ gastroscopy examination videos based on their experience in gastroscopy. The evaluation standard applied in the examination was the scale developed jointly by experts, including forward operation of gastroscopy, withdrawal observation, patients’ comfort and satisfaction, overall gastroscopy examination time and fluency, clarity of collected images, and diagnostic accuracy. We removed the highest score and the lowest score and obtained the average final score. The grading table is provided as Multimedia Appendix 2.
Gastroscope Roaming System Satisfaction Questionnaire
The reaction level was assessed by a satisfaction questionnaire. We used the questionnaire developed by Wang [35,36], which has been widely cited in evaluations of e-learning satisfaction [23]. This questionnaire has been shown to reach a reliability (Cronbach alpha) of .95 [35]. As the current version of the GRS is asynchronous, we did not use the learning community quality of Wang’s questionnaire. With the exception of the last two questions about overall satisfaction and success, we divided the remaining 22 questions into four modules: content, interface, testing, and personalization. We translated the questionnaire into Chinese and used a 5-point Likert scale to estimate each question anchored with strongly disagree to strongly agree and then counted the total points [37]. All the students in the EG completed the satisfaction questionnaire after completing their study. The questionnaire is provided as Multimedia Appendix 3.
Statistical Analysis
IBM SPSS version 24 was used for data analysis. The analysis included simple frequencies and descriptive analyses (means and standard deviations). First, the interaction of variables was verified by a general linear model. A paired sample t test was used to compare the test scores before and after the intervention in each group, and an independent sample t test was used to compare the pretest, posttest, and gastroscopy examination scores between the EG and the CG. The differences in the sex ratio and grade composition of the participants were analyzed by the chi-square test. For all the statistical analyses, we considered the significance level to be .05. Data analysis was used to determine whether there was a significant difference between the two experimental groups.
Results
Sample Description
Of the 44 masters degree students from the Second Affiliated Hospital of HMU who were assessed for eligibility, 42 (95%) who met the inclusion criteria completed the pretest and were included in the study. Two participants were excluded because one failed to complete the pretest and the other had a pretest score that was too high (>90%). Of the remaining participants, 21 were assigned to the EG and 21 to the CG following the principles of randomized controlled trials. During the learning period, only 1 student in the EG dropped out of the course. Therefore, data for 20 students in the EG and 21 students in the CG were analyzed. All 20 of the remaining students in the EG completed the satisfaction questionnaire. The baseline characteristics were similar in both the groups. Table 1 summarizes the characteristics of the participants.
Table 1.
Demographic and other characteristics of participants.
| Baseline characteristics | Electronic learning group (n=20) | Control group (n=21) | P value | |
| Age (years) | .77 | |||
|
|
Range | 23-27 | 23-26 |
|
|
|
Mean | 24.05 | 24.14 |
|
| Gender, n (%) | .39 | |||
|
|
Male | 6 (30) | 9 (43) |
|
|
|
Female | 14 (70) | 12 (57) |
|
| Current year of residency training, n (%) | .44 | |||
|
|
Postgraduate year 1 | 12 (60) | 15 (71) |
|
|
|
Postgraduate year 2 | 8 (40) | 6 (29) |
|
| Pretest score | .96 | |||
|
|
Range | 40-60 | 38-62 |
|
|
|
Mean | 50.86 | 50.76 |
|
Statistics and Analysis of Examination Scores
The mean pretest score (SD) in the EG was 50.86 (SD 6.11), and it improved to 83.70 (SD 5.99) in the posttest (P<.001). In contrast, the mean pretest score (SD) in the CG was 50.76 (SD 6.88), and it increased to 78.76 (SD 7.58) in the posttest (P<.001). After using the general linear model to analyze the difference between the EG and CG in the pretest and posttest, no interaction was found between the between-group variable and the within-group variable (P=.11). Therefore, a t test was used for further analysis. The results showed that there were no significant differences between the EG and CG (P=.96) in the pretest. After 1 month of learning, the posttest results of the EG were better than those of the CG (P=.03), and the test scores of the EG improved more than those of the CG (P=.01).
Regarding the gastroscopy examination scores, the mean score (SD) was 91.05 (SD 4.58) in the EG and 84.38 (SD 5.19) in the CG (P<.001).
User Satisfaction With the Gastroscope Roaming System
The overall satisfaction rate with the e-learning course was 77.0% (308/400). Only 11.5% (46/400) of the students were biased against e-learning, and the remaining 11.5% (46/400) maintained a neutral attitude. The last two overall satisfaction problems showed that 85% (17/20) of the participants were satisfied with the GRS, and 95% (19/20) of the participants thought it was successful. The satisfaction level for each problem is detailed in Figure 3.
Figure 3.
Satisfaction analysis of the system by Gastroscope Roaming System participants in four modules: (A) content quality, (B) interface quality, (C) testing quality, and (D) personalization quality. Each module lists the problems it contains. e-learning: electronic learning.
Regarding the content quality (Question 1-Question 4), 80% (64/80) of the learners thought that the course fit their needs and was sufficient, useful, and up-to-date. Regarding the interface quality (Question 5- Question 10), 73.3% (88/120) of the participants found the e-learning course to be stable, user friendly, and fast and agreed that it was easy to find the content needed. Regarding the testing quality (Question 11-Question 15), 80% (80/100) of the learners assessed the testing of the course as fair, secure, prompt, easy to understand, and easy to evaluate. Regarding the personalization quality (Question 16-Question 20), 76% (76/100) of the students thought that the GRS enabled them to learn the content needed, choose what to learn, control their learning progress, record their learning progress, and provide personalized learning support. Hence, there were more oppositional responses to the interface and personalization qualities than to the other qualities.
Discussion
Principal Findings
The first main outcome of this study was the development of an e-learning software named Gastroscope Roaming System for primary learners. To date, the conventional gastroscopy teaching mode has been face-to-face tutorials. The number of learners is, therefore, very small; the instruction is lengthy; and it is not convenient to practice only on patients. Textbooks and handouts appear to be the extensive way of disseminating knowledge in education [38]. However, for students in the field of medical operations, especially endoscopy teaching, traditional lectures and spectrograms are not conducive to learning how to operate and gain experience effectively. In addition, gastroscopy simulators have problems of high cost, limited teaching resources, and low popularity. As constructed in this experiment, an internet-based e-learning system offers tremendous advantages over the traditional teaching methods because there are no limits on the number of learners or learning time and place. Students can study gastroscopy with their laptop at any time and place instead of being confined to an observational study in the narrow gastroscopy room, thus enabling them to more reasonably allocate and use their study time. Gastroscopy learning has three aspects: knowledge, skills, and experience. Through conventional lectures or hands-on seminars, endoscopists can acquire only the relevant knowledge and techniques [39]. Therefore, we collected many endoscopic images of common UGI diseases and inserted them into the software for primary learners to accumulate experience. Given the creation of the new gastroscope learning method, we can expect to apply this method to more endoscopic learning and training.
The second principal result was the comparison of the learning results of the GRS-based EG with those of the traditional learning group, showing that e-learning can improve the learning efficiency of primary gastroscopists. Notably, the EG in this experiment used a combination of e-learning and traditional learning, that is, blended learning, which seemed to be more effective than single e-learning with respect to knowledge acquisition. Although e-learning may not be an alternative to traditional learning in some cases, it can provide supplements to traditional education and is a useful complement to traditional education [40-42].
Before gastroscopy learning, there was no significant difference between the two groups in the predicted test, indicating that the level of the participants was similar. After 1 month of study, the theoretical test scores of all the students significantly improved, and the score of the EG increased slightly more than that of the CG. However, the gastroscopy examination score of the EG was notably higher than that of the CG after 1 month of learning, indicating that e-learning is more effective than traditional learning, especially in improving students’ gastroscopic handling ability. The theoretical-level test verified the participants’ basic knowledge of gastroscopy and their ability to diagnose disease, whereas the gastroscopy examination verified their practical ability. In other words, gastroscopy ability=cognitive ability+practical ability. The technical level is most important for beginners learning gastroscopy. Therefore, we believe that the GRS is essential for helping students master gastroscopy skills in a short time. The GRS provides learners with a virtual UGI space, which enables them to explore the structure of the UGI freely and then, through repeated learning, to master and familiarize themselves with the structure of the UGI. In the process of a real gastroscopy procedure, they can operate skillfully and confidently, accelerate their learning speed, and avoid the harm caused to patients by confusion and fear. Moreover, by repeatedly studying the endoscopic pictures and characteristics of common diseases shown in the GRS, students can learn to accurately identify lesions in actual gastroscopy.
The third main result was that the satisfaction questionnaire showed that the overall satisfaction rate of the students in the EG was very high. Using Wang’s questionnaire, we determined how satisfied the students were with each part. We found that the students were more satisfied with the content and testing qualities and less satisfied with the interface and personalization qualities. In terms of the interface quality, students found that the operational interface of the software was somewhat complex, difficult to understand, and not smooth. This response may be because of the software design using the W/A/S/D keys on keyboard with mouse movement. The students had different proficiency levels in computer operation and different computer configurations.
Limitations
Our study also has limitations. For example, the small sample size in a single institution affects the generalizability of the results. The main reason was that our GRS software was in the early stage of construction. We will continue to improve, upgrade, and expand its content in the future. Therefore, the software was applied only on a small scale in the hospital with which the author is affiliated to assess the efficiency and effectiveness of the e-learning model. Regarding the assessment of the real gastroscopy procedure, because of differences in the physiological structure of the UGI and the pain and tolerance levels of each patient, it was not guaranteed that each student would face the same difficulty in the assessment. Even with three opportunities, the participants might not have been able to perform at the real level of gastroscopy, although the probability of this problem is low. Finally, we should carefully consider the validity of the satisfaction assessment. We used an unproven Chinese version of the user satisfaction questionnaire because we could not find a Chinese-validated translation for e-learning systems in the literature.
Conclusions
The results of our study suggest that e-learning is an effective educational strategy for primary learners to acquire skills in gastroscopy examination and master the characteristics of endoscopic images of the UGI. The GRS may serve as a meaningful supplementary approach to on-campus endoscopy education. The novel computer assistant module in the field of endoscopic teaching will strengthen the training of skilled endoscopists. We will continue to upgrade and improve the GRS system to enhance the authenticity of the simulation and to enrich the variety of diseases. In summary, e-learning can be widely used as an effective assistant to supplement routine teaching and learning in medical education.
Acknowledgments
This study was funded by the China Medical Board (CMB; project number 15-238). The authors are grateful to HMU for training the authors and to the leaders, teachers, and graduate students who helped with this study. The authors sincerely thank the CMB for its support and the animation faculty, computer center, and students of Harbin University of Science and Technology for their technical cooperation.
Abbreviations
- CG
non–e-learning control group
- CMB
China Medical Board
- e-learning
electronic learning
- EG
e-learning group
- GI
gastrointestinal
- GRS
Gastroscope Roaming System
- HMU
Harbin Medical University
- UGI
upper gastrointestinal
Appendix
Multiple-choice question test paper for evaluating Gastroscope Roaming System learning efficiency.
Grading table for gastroscopy examination.
Gastroscope Roaming System satisfaction questionnaire.
CONSORT-EHEALTH (V 1.6.1) -Submission/Publication Form.
Footnotes
Authors' Contributions: SJ and SL contributed to study concept and design. SL, GL, YL, and WX contributed to trial implementation and supervision. SL, NY, HC, NL, and KL contributed to acquisition of data and statistical analysis. SJ and SL drafted the manuscript. SJ contributed to study supervision. All authors contributed to interpretation of the results, critical revision of the manuscript for important intellectual content, and final approval of the manuscript.
Conflicts of Interest: None declared.
References
- 1.Ekkelenkamp VE, Koch AD, de Man RA, Kuipers EJ. Training and competence assessment in GI endoscopy: a systematic review. Gut. 2016 Apr;65(4):607–15. doi: 10.1136/gutjnl-2014-307173. [DOI] [PubMed] [Google Scholar]
- 2.ASGE Technology Committee. Goodman AJ, Melson J, Aslanian HR, Bhutani MS, Krishnan K, Lichtenstein DR, Navaneethan U, Pannala R, Parsi MA, Schulman AR, Sethi A, Sullivan SA, Thosani N, Trikudanathan G, Trindade AJ, Watson RR, Maple JT. Endoscopic simulators. Gastrointest Endosc. 2019 Jul;90(1):1–12. doi: 10.1016/j.gie.2018.10.037. [DOI] [PubMed] [Google Scholar]
- 3.Khan R, Plahouras J, Johnston BC, Scaffidi MA, Grover SC, Walsh CM. Virtual reality simulation training in endoscopy: a Cochrane review and meta-analysis. Endoscopy. 2019 Jul;51(7):653–64. doi: 10.1055/a-0894-4400. [DOI] [PubMed] [Google Scholar]
- 4.Thorne CJ, Lockey AS, Bullock I, Hampshire S, Begum-Ali S, Perkins GD, Advanced Life Support Subcommittee of the Resuscitation Council (UK) E-learning in advanced life support--an evaluation by the Resuscitation Council (UK) Resuscitation. 2015 May;90:79–84. doi: 10.1016/j.resuscitation.2015.02.026. [DOI] [PubMed] [Google Scholar]
- 5.Taveira-Gomes T, Saffarzadeh A, Severo M, Guimarães MJ, Ferreira MA. A novel collaborative e-learning platform for medical students - ALERT STUDENT. BMC Med Educ. 2014 Jul 14;14:143. doi: 10.1186/1472-6920-14-143. https://bmcmededuc.biomedcentral.com/articles/10.1186/1472-6920-14-143. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Holzinger A, Kickmeier-Rust MD, Wassertheurer S, Hessinger M. Learning performance with interactive simulations in medical education: Lessons learned from results of learning complex physiological models with the HAEMOdynamics SIMulator. Comput Educ. 2008;52(2):292–301. doi: 10.1016/j.compedu.2008.08.008. [DOI] [Google Scholar]
- 7.Obdeijn MC, Bavinck N, Mathoulin C, van der Horst CM, Schijven MP, Tuijthof GJ. Education in wrist arthroscopy: past, present and future. Knee Surg Sports Traumatol Arthrosc. 2015 May;23(5):1337–45. doi: 10.1007/s00167-013-2592-y. [DOI] [PubMed] [Google Scholar]
- 8.Kim K, Han J, Park IB, Kee C. Medical education in Korea: the e-learning consortium. Med Teach. 2009 Sep;31(9):e397–401. doi: 10.1080/01421590902744902. [DOI] [PubMed] [Google Scholar]
- 9.Sajeva M. E-learning: web-based education. Curr Opin Anaesthesiol. 2006 Dec;19(6):645–9. doi: 10.1097/ACO.0b013e328010bec8. [DOI] [PubMed] [Google Scholar]
- 10.Lahti M, Hätönen H, Välimäki M. Impact of e-learning on nurses' and student nurses knowledge, skills, and satisfaction: a systematic review and meta-analysis. Int J Nurs Stud. 2014 Jan;51(1):136–49. doi: 10.1016/j.ijnurstu.2012.12.017. [DOI] [PubMed] [Google Scholar]
- 11.Feng JY, Chang H, Chang HY, Erdley WS, Lin CH, Chang YJ. Systematic review of effectiveness of situated e-learning on medical and nursing education. Worldviews Evid Based Nurs. 2013 Aug;10(3):174–83. doi: 10.1111/wvn.12005. [DOI] [PubMed] [Google Scholar]
- 12.Perkins GD, Fullerton JN, Davis-Gomez N, Davies RP, Baldock C, Stevens H, Bullock I, Lockey AS. The effect of pre-course e-learning prior to advanced life support training: a randomised controlled trial. Resuscitation. 2010 Jul;81(7):877–81. doi: 10.1016/j.resuscitation.2010.03.019. [DOI] [PubMed] [Google Scholar]
- 13.Ruiz JG, Mintzer MJ, Leipzig RM. The impact of E-learning in medical education. Acad Med. 2006 Mar;81(3):207–12. doi: 10.1097/00001888-200603000-00002. [DOI] [PubMed] [Google Scholar]
- 14.Beux PL, Fieschi M. Virtual biomedical universities and e-learning. Int J Med Inform. 2007;76(5-6):331–5. doi: 10.1016/S1386-5056(07)00060-3. [DOI] [PubMed] [Google Scholar]
- 15.Wutoh R, Boren SA, Balas EA. eLearning: a review of internet-based continuing medical education. J Contin Educ Health Prof. 2004;24(1):20–30. doi: 10.1002/chp.1340240105. [DOI] [PubMed] [Google Scholar]
- 16.Wong G, Greenhalgh T, Pawson R. Internet-based medical education: a realist review of what works, for whom and in what circumstances. BMC Med Educ. 2010 Feb 2;10:12. doi: 10.1186/1472-6920-10-12. https://bmcmededuc.biomedcentral.com/articles/10.1186/1472-6920-10-12. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17.Chumley-Jones HS, Dobbie A, Alford CL. Web-based learning: sound educational method or hype? A review of the evaluation literature. Acad Med. 2002 Oct;77(10 Suppl):S86–93. doi: 10.1097/00001888-200210001-00028. [DOI] [PubMed] [Google Scholar]
- 18.Cook DA, Garside S, Levinson AJ, Dupras DM, Montori VM. What do we mean by web-based learning? A systematic review of the variability of interventions. Med Educ. 2010 Aug;44(8):765–74. doi: 10.1111/j.1365-2923.2010.03723.x. [DOI] [PubMed] [Google Scholar]
- 19.Abdelhai R, Yassin S, Ahmad MF, Fors UG. An e-learning reproductive health module to support improved student learning and interaction: a prospective interventional study at a medical school in Egypt. BMC Med Educ. 2012 Mar 20;12:11. doi: 10.1186/1472-6920-12-11. https://bmcmededuc.biomedcentral.com/articles/10.1186/1472-6920-12-11. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20.Moreira IC, Ventura SR, Ramos I, Rodrigues PP. Development and assessment of an e-learning course on breast imaging for radiographers: a stratified randomized controlled trial. J Med Internet Res. 2015 Jan 5;17(1):e3. doi: 10.2196/jmir.3344. https://www.jmir.org/2015/1/e3/ [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21.Maloney S, Chamberlain M, Morrison S, Kotsanas G, Keating JL, Ilic D. Health professional learner attitudes and use of digital learning resources. J Med Internet Res. 2013 Jan 16;15(1):e7. doi: 10.2196/jmir.2094. https://www.jmir.org/2013/1/e7/ [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22.Pinto A, Brunese L, Pinto F, Acampora C, Romano L. E-learning and education in radiology. Eur J Radiol. 2011 Jun;78(3):368–71. doi: 10.1016/j.ejrad.2010.12.029. [DOI] [PubMed] [Google Scholar]
- 23.Oliveira AC, Mattos S, Coimbra M. Development and assessment of an E-learning course on pediatric cardiology basics. JMIR Med Educ. 2017 May 10;3(1):e10. doi: 10.2196/mededu.5434. https://mededu.jmir.org/2017/1/e10/ [DOI] [PMC free article] [PubMed] [Google Scholar]
- 24.Ward JP, Gordon J, Field MJ, Lehmann HP. Communication and information technology in medical education. Lancet. 2001 Mar 10;357(9258):792–6. doi: 10.1016/S0140-6736(00)04173-8. [DOI] [PubMed] [Google Scholar]
- 25.Dhir SK, Verma D, Batta M, Mishra D. E-learning in medical education in India. Indian Pediatr. 2017 Oct 15;54(10):871–7. doi: 10.1007/s13312-017-1152-9. https://www.indianpediatrics.net/oct2017/871.pdf. [DOI] [PubMed] [Google Scholar]
- 26.Nakanishi H, Doyama H, Ishikawa H, Uedo N, Gotoda T, Kato M, Nagao S, Nagami Y, Aoyagi H, Imagawa A, Kodaira J, Mitsui S, Kobayashi N, Muto M, Takatori H, Abe T, Tsujii M, Watari J, Ishiyama S, Oda I, Ono H, Kaneko K, Yokoi C, Ueo T, Uchita K, Matsumoto K, Kanesaka T, Morita Y, Katsuki S, Nishikawa J, Inamura K, Kinjo T, Yamamoto K, Yoshimura D, Araki H, Kashida H, Hosokawa A, Mori H, Yamashita H, Motohashi O, Kobayashi K, Hirayama M, Kobayashi H, Endo M, Yamano H, Murakami K, Koike T, Hirasawa K, Miyaoka Y, Hamamoto H, Hikichi T, Hanabata N, Shimoda R, Hori S, Sato T, Kodashima S, Okada H, Mannami T, Yamamoto S, Niwa Y, Yashima K, Tanabe S, Satoh H, Sasaki F, Yamazato T, Ikeda Y, Nishisaki H, Nakagawa M, Matsuda A, Tamura F, Nishiyama H, Arita K, Kawasaki K, Hoppo K, Oka M, Ishihara S, Mukasa M, Minamino H, Yao K. Evaluation of an e-learning system for diagnosis of gastric lesions using magnifying narrow-band imaging: a multicenter randomized controlled study. Endoscopy. 2017 Oct;49(10):957–67. doi: 10.1055/s-0043-111888. [DOI] [PubMed] [Google Scholar]
- 27.Alliger GM, Janak EA. Kirkpatrick's levels of training criteria: thirty years later. Pers Psychol. 1989 Jun;42(2):331–42. doi: 10.1111/j.1744-6570.1989.tb00661.x. [DOI] [Google Scholar]
- 28.Kirkpatrick D. Great ideas revisited. Techniques for evaluating training programs. Revisiting Kirkpatrick's Four-Level Model. Train Dev. 1996;50(1):54–9. https://eric.ed.gov/?id=EJ515660. [Google Scholar]
- 29.Galloway DL. Evaluating distance delivery and e-learning is Kirkpatrick's model relevant? Perform Improv. 2005 Apr;44(4):21–7. doi: 10.1002/pfi.4140440407. [DOI] [Google Scholar]
- 30.Santos G, Leite A, Figueiredo P, Pimentel N, Flores-Mir C, de Melo NS, Guerra EN, Canto GL. Effectiveness of E-learning in oral radiology education: a systematic review. J Dent Educ. 2016 Sep;80(9):1126–39. http://www.jdentaled.org/cgi/pmidlookup?view=long&pmid=27587580. [PubMed] [Google Scholar]
- 31.Carriero A, Bonomo L, Calliada F, Campioni P, Colosimo C, Cotroneo A, Cova M, Ettorre GC, Fugazzola C, Garlaschi G, Macarini L, Mascalchi M, Meloni GB, Midiri M, Mucelli RP, Rossi C, Sironi S, Torricelli P, Beomonte BZ, Zompatori M, Zuiani C. E-learning in radiology: an Italian multicentre experience. Eur J Radiol. 2012 Dec;81(12):3936–41. doi: 10.1016/j.ejrad.2012.07.007. [DOI] [PubMed] [Google Scholar]
- 32.Yao K, Uedo N, Muto M, Ishikawa H, Cardona HJ, Filho EC, Pittayanon R, Olano C, Yao F, Parra-Blanco A, Ho SH, Avendano AG, Piscoya A, Fedorov E, Bialek AP, Mitrakov A, Caro L, Gonen C, Dolwani S, Farca A, Cuaresma LF, Bonilla JJ, Kasetsermwiriya W, Ragunath K, Kim SE, Marini M, Li H, Cimmino DG, Piskorz MM, Iacopini F, So JB, Yamazaki K, Kim GH, Ang TL, Milhomem-Cardoso DM, Waldbaum CA, Carvajal WA, Hayward CM, Singh R, Banerjee R, Anagnostopoulos GK, Takahashi Y. Development of an E-learning system for the endoscopic diagnosis of early gastric cancer: an international multicenter randomized controlled trial. EBioMedicine. 2016 Jul;9:140–7. doi: 10.1016/j.ebiom.2016.05.016. https://linkinghub.elsevier.com/retrieve/pii/S2352-3964(16)30199-2. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 33.Stark CM, Graham-Kiefer ML, Devine CM, Dollahite JS, Olson CM. Online course increases nutrition professionals' knowledge, skills, and self-efficacy in using an ecological approach to prevent childhood obesity. J Nutr Educ Behav. 2011;43(5):316–22. doi: 10.1016/j.jneb.2011.01.010. [DOI] [PubMed] [Google Scholar]
- 34.Hashemikamangar SS, Yazdanpanah F, Mirzaii M, Yazdani R, Karazifard MJ, Yasini E. Efficacy of E-learning via the website of Tehran University of Medical Sciences for diagnosing tooth discolorations and treatment planning by senior dental students. Acta Med Iran. 2016 Aug;54(8):536–41. http://acta.tums.ac.ir/index.php/acta/article/view/5123. [PubMed] [Google Scholar]
- 35.Wang YS. Assessment of learner satisfaction with asynchronous electronic learning systems. Inf Manag. 2003 Oct;41(1):75–86. doi: 10.1016/S0378-7206(03)00028-4. doi: 10.1016/S0378-7206(03)00028-4. [DOI] [Google Scholar]
- 36.AL-Smadi M, Wesiak G, Guetl C, Holzinger A. Assessment for/as Learning: Integrated Automatic Assessment in Complex Learning Resources for Self-Directed Learning. Proceedings of the 2012 Sixth International Conference on Complex, Intelligent, and Software Intensive Systems; CISIS'12; July 4-6, 2012; Palermo, Italy. 2012. pp. 929–34. [DOI] [Google Scholar]
- 37.Hersey P, McAleer S. Developing an e-learning resource for nurse airway assistants in the emergency department. Br J Nurs. 2017 Feb 23;26(4):217–21. doi: 10.12968/bjon.2017.26.4.217. [DOI] [PubMed] [Google Scholar]
- 38.Trelease RB. From chalkboard, slides, and paper to e-learning: How computing technologies have transformed anatomical sciences education. Anat Sci Educ. 2016 Nov;9(6):583–602. doi: 10.1002/ase.1620. [DOI] [PubMed] [Google Scholar]
- 39.Baumgart DC, Wende I, Grittner U. Tablet computer-based multimedia enhanced medical training improves performance in gastroenterology and endoscopy board style exam compared with traditional medical education. Gut. 2016 Mar;65(3):535–6. doi: 10.1136/gutjnl-2015-309895. [DOI] [PubMed] [Google Scholar]
- 40.Zhan X, Zhang Z, Sun F, Liu Q, Peng W, Zhang H, Yan W. Effects of improving primary health care workers' knowledge about public health services in rural China: a comparative study of blended learning and pure E-learning. J Med Internet Res. 2017 May 1;19(5):e116. doi: 10.2196/jmir.6453. https://www.jmir.org/2017/5/e116/ [DOI] [PMC free article] [PubMed] [Google Scholar]
- 41.Bollinger R, Chang H, Jafari R, O'Callaghan T, Ngatia P, Settle D, McKenzie-White J, Patel K, Dossal A, Al Shorbaji N. Leveraging information technology to bridge the health workforce gap. Bull World Health Organ. 2013 Nov 1;91(11):890–2. doi: 10.2471/BLT.13.118737. http://europepmc.org/abstract/MED/24347719. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 42.Liu Q, Peng W, Zhang F, Hu R, Li Y, Yan W. The effectiveness of blended learning in health professions: systematic review and meta-analysis. J Med Internet Res. 2016 Jan 4;18(1):e2. doi: 10.2196/jmir.4807. https://www.jmir.org/2016/1/e2/ [DOI] [PMC free article] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Supplementary Materials
Multiple-choice question test paper for evaluating Gastroscope Roaming System learning efficiency.
Grading table for gastroscopy examination.
Gastroscope Roaming System satisfaction questionnaire.
CONSORT-EHEALTH (V 1.6.1) -Submission/Publication Form.