Abstract
Background/purpose
The SimEx is dental training system that applies new technology in a computerized dental simulator. The purpose of this study was to understand the usability satisfaction of the SimEx dental education and evaluation system by dental students and dentists at Tohoku University.
Materials and methods
In this study, the Tohoku University IRB execution number was 2020-3-33. The number of subjects accepted was 59 at Tohoku University and divided into 4 groups based on years of clinical experience (Group A: 0 years; Group B: 1–2 years; Group C: 2–5 years; Group D: at least 5 years), and a total of 58 usability questionnaires were collected. Subjects completed the SimEx Usability Satisfaction Questionnaire after operating the SimEx (EPED Inc., Kaohsiung, Taiwan) course, which contained 16 questions.
Results
Among the 58 questionnaires collected by Tohoku University, there were 19 undergraduate students (4th∼6th grade), 12 post-graduate students, 14 residents, and 13 dentists. Significant differences between Group A and Group B, and between Group A and Group D were found (P < 0.05). The same results were obtained for the “experience satisfaction index”. In the items where significant differences were found, longer clinical experience tended to result in lower scores.
Conclusion
From these results, we can conclude that the SimEx education and evaluation system facilitates students' self-learning, and this system is very useful for continued study and clinical skill training for dentists, especially for students and junior dentists with high usability satisfaction.
Keywords: Dental education, SimEx, Self-assessment, Questionnaire survey, Computer-based technology
Introduction
Dental education is mainly composed of three parts, that is the lectures (PBL learning) part, the simulation laboratory course, and the clinical skill training. Of these three parts, the simulation laboratory course and clinical training are the most important infrastructure of preclinical learning in dental education. Because simulation-based clinical skill training is the first step in becoming a good clinical practitioner.1 Since the phantom head simulator was introduced to clinical skill training of dental education in 1894,2 simulation training systems have been improved in many ways to improve the effectiveness of dental education, especially clinical skills training, and are currently used in many dental education settings.2 Since the late 1990s, development toward dental education applications of virtual reality (VR) and computer-supported simulators has progressed, and in the early 2000s, simulation systems using new technologies, such as DentSim, began to appear and to be used in dental education.2
The simulation-based clinical skill training includes briefing, simulation, feedback, debriefing, reflection, and evaluation phase. Each of the above phases is interdependent and all of these phases must be performed to complete the learning loop.3 These sessions are very efficient for testing the dental student's competence in a clinical setting, either with a custom design or with slight modifications to an existing methodology.3 During the learning loop, providing effective feedback is an important component that allows for timely and specific corrections, as well as identifying types of errors, recognizing challenges, and making recommendations for better results.4 A more effective approach is to use a form of communication such as verbal, written, graphic, or video to raise the issue, mention the mistake, and finally offer a solution. Hajhamid et al.5 reported that if dental school curricula included teaching feedback methods, it could improve the outcomes of students' clinical and laboratory activities and help teachers receive constructive comments and adjust their teaching methods.
Another important part of improving the educational outcome for dental students is self-assessment.6 The main purpose of assessment is to optimize the student's ability by providing motivation and direction for future learning. In this context, self-assessment is a way to enhance the student's role as an active participant in his/her learning by involving the student themselves in determining whether they have met the set standards.7 Self-assessment skills are essential to safe and effective practice and can be used to promote lifelong learning. Furthermore, self-assessment is an important method in education, as it can be an effective learning tool.8,9 Burrows7 suggests that SA should be a collaborative process, with teachers and students developing reflective skills through mutual feedback.
In recent years, with the advanced development of Information Technology (IT), dental education tools using computer-based technologies such as VR and augmented reality (AR) have been attracting attention, and their use in dental education has been attempted.10,11 It has been reported that systems using VR and AR can be educational tools that allow students to learn on their own as well as integrate systems for learning and teaching, training skills, and improving hand-eye coordination.10,11 Through a comparison of the two dental simulation systems that is VR and contemporary non-computer assisted, Jasinevicius et al.12 suggest that virtual reality technology has the potential to provide an efficient and more self-directed approach to learning clinical psychomotor skills. Gottlieb et al.13 also suggest that virtual reality simulation (VRS)-trained students may have an advantage in the clinical setting compared to students without VRS training. Therefore, modern technology has positively influenced dental education.14 However, the report of Berry et al.15 showed that the use of technology-enhanced learning (TEL) alone did not significantly improve periodontics education outcomes compared to traditional learning methods, highlighting the need for a combination of TEL and traditional education methods.
Several VR and AR-based simulators are in partial use in dental education right now, such as PerioSim, iDental, Simodont Dental Trainer, Voxel-Man Simulator, and so on, but they still have their disadvantages and leave room for improvement. The current system has hardware disadvantages such as inadequate stereoscopic viewing and resolution of the display, no fixed physical finger rest, and lack of two-handed coordinated operation.16 Furthermore, there are software disadvantages as well, such as force feedback simulations that are not realistic enough, insufficient training content, and inaccurate evaluation of training results.16
The SimEx is another dental training system that applies new technology in a computerized dental simulator developed by EPED Inc.10 This technology can provide the best-computerized training system for dental students and dentists who need self-training. The optical positioning system that makes up this simulator provides accurate 3D real-time feedback on the optimal tooth angle and depth, and the extensive software lessons (dental surgery, endodontics, crown & bridge, pediatric dentistry) provide students with digital guidance and simulation for easy self-study and practice.10 Courses and lessons can be customized, designed, and upgraded for specific projects. This simulator also allows for objective computer-based assessment, and teachers can easily set and highlight scoring percentages.10 In addition, digital reports allow students to assess their progress and better ensure that they are achieving their learning goals. It is said that evaluation reports with numerical values and explanations can facilitate self-learning and comparisons, and can also improve the accuracy of clinical practice. In addition, the recorded information can be used to check progress and identify errors, thereby improving skills.10 In the classroom, the broadcast function allows you to easily give presentations and demonstrations to remotely connected students. Through the digital dental simulator and clinical environment, students can easily self-practice and expect to gain important clinical experience and accuracy.
The purpose of this study was to understand the usability satisfaction of the SimEx dental education and evaluation system by dental students and doctors at Tohoku University, Japan.
Materials and methods
This study was approved by the IRB of Tohoku University, the IRB execution number was 2020-3-33. In this study, 59 subjects were asked to complete and return a structured questionnaire (Fig. 1) after finishing the training Class I cavity for amalgam twice using SimEx Dental Education and Evaluation System (EPED Inc., Kaohsiung, Taiwan). The subjects were selected at random from fourth-year, fifth-year, and sixth-year undergraduate dental students, post-graduate students, residents, and dentists at the School of Dentistry, Graduate School of Dentistry, and University Hospital (Tohoku University, Sendai, Japan). They included 19 dental students (6 fourth-year, 7 fifth-year, and 7 sixth-year; men, 4; women, 15), 13 post-graduate students (men, 11; women, 2), 14 residents (men, 3; women, 11), and 13 dentists (men, 11; women, 2). The mean ages of the dental students and dentists were 24.5, 25.9, 27.5, and 38.2 years, respectively. The distribution and collection of the questionnaire were instituted by the members of this survey in 2022. This questionnaire was administered after explaining the aim of the survey to the subjects and gaining their consent.
Figure 1.
Questionnaire survey sheet.
The questionnaire was written in English and then translated into Japanese. The questionnaire consists of 16 questions: eight questions about the hardware and software satisfaction index, and eight questions about the experience satisfaction index. We are using a Likert scale, a type of psychometric response scale widely used in various surveys such as questionnaires, each question had five response options; “Very dissatisfied” (score 1), “Not satisfied” (score 2), “Neutral” (score 3), “Satisfied” (score 4), and “Very satisfied” (score 5).
All subjects were divided into 4 groups based on years of clinical experience, that is Group-A (0 years; n: 19), Group B (1–2 years; n: 20), Group C (2–5 years; n: 7), and Group D (at least 5 years; n: 13). Mann-Whitney U tests were used to examine the differences in the responses for each item on the questionnaire between each group. All analyses were computed with IBM SPSS Statistics (Ver. 25, IBM Japan, Tokyo, Japan).
Results
The response rate was 98.3%. The overall satisfaction score for hardware devices was 3.79 ± 0.18, and the software stability is 3.79 ± 0.15. The overall user experience satisfaction of SimEx is 4.10 ± 0.28 (Table 1). Fig. 2 shows the statistical analysis results of the “hardware and software satisfaction index”. Significant differences between Group A and Group B, and between Group A and Group D were found (P < 0.05) (Table 1). The same results were obtained for the “experience satisfaction index” (Fig. 3) (Table 1). When the comparison was limited to the “software satisfaction index” alone, the tendency was to find a predominant difference between those with shorter and longer clinical experience (Fig. 4) (Table 1). No differences were observed between groups for the “hardware satisfaction index” (Fig. 5) (Table 1). The results of the individual comparisons of the eight questions on the “experience satisfaction index” tended to show significance according to the length of clinical experience for question 2 “Training record and shortcoming improvement”, question 3 “Improve the Clinical operation accuracy”, and question 5 “Clinical confidence boost after SimEx experience” (Figure 6, Figure 7). In the items where significant differences were found, longer clinical experience tended to result in lower scores.
Table 1.
Average score of each index.
| Question | Group |
||||
|---|---|---|---|---|---|
| Group A | Group B | Group C | Group D | Overall Score | |
| Hardware and software satisfaction index | 33.16 (4.51) | 28.10 (6.30) | 26.50 (8.22) | 28.85 (4.87) | 3.64 (0.18) |
| Experience satisfaction index | 36.58 (3.15) | 33.15 (4.37) | 29.00 (9.07) | 32.38 (5.46) | 4.10 (0.28) |
| Software satisfaction index | 16.89 (2.55) | 13.70 (3.66) | 11.83 (4.52) | 13.62 (2.73) | 3.50 (0.20) |
| Hardware satisfaction index | 16.26 (2.90) | 14.40 (2.85) | 14.67 (4.15) | 15.23 (2.66) | 3.79 (0.15) |
∗Average score with standard deviation.
Figure 2.
Statistical analysis results of “hardware and software satisfaction index”.
Figure 3.
Statistical analysis results of “experience satisfaction index”.
Figure 4.
Statistical analysis results of “software satisfaction index”.
Figure 5.
Statistical analysis results of “hardware satisfaction index”.
Figure 6.
Statistical analysis results of individual comparisons of each question on the “experience satisfaction index”-Questions 1–4.
Figure 7.
Statistical analysis results of individual comparisons of each question on the “experience satisfaction index”-Questions 5–8.
For all questions, older age groups tended to score lower than young age groups.
Discussion
Since 1984, phantom head simulator was used to clinical skill training of dental education, the simulator which used in dental education has developed rapidly. Since the early 1900s, the combination of bench-top and phantom head simulator for the acquisition of dental clinical skill has become mainstream and is still used in dental schools around the world in educational settings.1,2 Since the late 1990s, research and development of dental simulators using IT technology has been active, and several related products have been used in educational settings. Recently, there have two basic type dental simulator been used in dental school, that is combination of bench-top and phantom head simulator (traditional simulator) and dental simulators using computer-based technologies (novel simulator). The SimEx dental education and evaluation system is an innovative dental simulator that combines traditional simulators with computer-based technology assistant simulator for self-training, computer-based assessment, and self-assessment.10
For the undergraduate students targeted in this study, we selected fourth-, fifth- and sixth-year students. Undergraduate dental education in Japan is a six-year course, the pre-clinical course will start from fourth year. This is the reason why we targeted students from the fourth year.
The student and doctors in Tohoku University has not used the SimEx dental education and evaluation system before, so the degree of proficiency in the operation is not familiar, and the young people are more comfortable with the computer control, the learning curve is shorter, they will feel interested, so it will affect the score. This may be the reason why young people have higher Scores than older people.
In the results of questions regarding “experience satisfaction index”, significant differences between Group A and Group B, and between Group A and Group D were found (P < 0.05). From the answers of Q2 and Q6, there was a difference in experience satisfaction about SimEx between student and junior dentistry, senior dentistry. Also, for Q3, the significant different was found between not only student and junior dentistry, but also between junior dentistry and senior dentistry (Figure 6, Figure 7). This suggests that SimEx education and evaluation system can record the training process and is excellent for objective evaluation and grading of training results. However, these advantages of SimEx are easily influenced by the years of clinical experience and may be more effective for students or dentists with less clinical experience. Although no significant differences were found for Q1 and Q4, all groups scored 4 or higher, indicating a positive evaluation of the SimEx (Table 2). This suggests that SimEx education and evaluation system is effective for self-learning and useful for curriculum diversification.
Table 2.
Results of the questions regarding “experience satisfaction index”.
| Question | Group |
|||
|---|---|---|---|---|
| Group A | Group B | Group C | Group D | |
| Q1: Improve the Self-learning experience | 4.47 (0.60) | 4.30 (0.56) | 4.00 (1.00) | 4.15 (0.86) |
| Q2: Training record and shortcoming improvement | 4.79 (0.41) | 4.30 (0.64) | 3.67 (1.25) | 4.31 (0.91) |
| Q3: Improve the Clinical operation accuracy | 4.74 (0.44) | 4.15 (0.79) | 3.17 (1.57) | 3.85 (1.17) |
| Q4: Content diversity | 4.32 (0.73) | 4.05 (0.74) | 4.50 (0.50) | 4.15 (0.77) |
| Q5: Clinical confidence boost after SimEx experience | 4.37 (0.58) | 4.05 (0.74) | 3.00 (1.53) | 3.92 (0.92) |
| Q6: Score objectivity | 4.79 (0.41) | 4.15 (0.73) | 3.83 (1.34) | 4.15 (0.95) |
| Q7: Overall experience with SimEx system | 4.42 (0.67) | 3.90 (0.70) | 3.17 (1.57) | 3.69 (1.20) |
| Q8: Is more effective for training than not using SimEx | 4.68 (0.57) | 4.25 (0.62) | 3.67 (1.60) | 4.15 (0.86) |
∗Average score with standard deviation.
In dental education, “skills training” is required for students to master and embody the use of instruments and equipment and their clinical skill improvement. The traditional teaching and evaluation in clinical skills training tended to rely on the subjectivity of the instructor, and there was a lack of learning support methods and objective, standardized evaluation methods that were tailored to each student's abilities and individuality.2,3 Effective feedback provision is an important factor of the learning loop of clinical skills training, including simulation, feedback, evaluation, and so on.4,5 Furthermore, effective feedback is also essential for effective self-assessment in improving student educational outcomes.7,8 The SimEx dental education and evaluation system can provide students with accurate 3D real-time feedback, digital guidance and simulation, as well as objective computer-based assessment, facilitating student self-study and self-assessment and improving clinical skills.10 These could be the reasons why for Q1 and Q4, all groups scored 4 or higher. Therefore, this system is promising for provide standardized assessments, and individualized, standardized, and diversified clinical skills education content, it is expected to lead to personalized learning optimized for individual students and realize quality assurance in clinical skills training of dental education. Furthermore, the use of SimEx system will enable students to self-learning and self-assessment, which is expected to contribute to the improvement of educational effects.5,6 This may be the reason for the significant differences between the groups in response to Q2, Q3 and Q6, and also the reason why students and junior dentistry scored higher than senior dentistry for Q2, Q3 and Q6.
Since the pandemic of coronavirus disease 2019 (COVID-19) occurred in 2020, most dental schools throughout the world either suspended or postponed their simulated and clinical skills training courses, most of education activity was switched to online mode to keep on the learning progress for dental students. However, while lecture part can easily be switched online, clinical skills training part is very difficult to switched full online. Therefore, many dental schools try to find new technological tools to solve this problem.17, 18, 19 The broadcasting capabilities of SimEx allow for easy presentations and demonstrations to remotely connected students. Therefore, this system would be one candidate for a useful tool for continuing dental education especially clinical skill training in a variety of settings.
Some limitations can be identified in this study. The sample size was small (only 59 subjects), no comparison was made between subjects using SimEx system and those with the traditional simulator system, and only one model case was used for training, and so on. It is unknown whether the results will generalize to other samples, therefore, to conduct future research in more diverse samples is necessary. In addition, a comparative study of the effectiveness of the traditional simulator system and the SimEx dental education and evaluation system will be conducted.
Within the limitation of this study, we can conclude that the usability satisfaction of the SimEx dental education and evaluation system is high among the students and junior dentists. And it can be concluded that the SimEx dental education and evaluation system facilitates students' self-learning, and this system is very useful for continued study and clinical skill training for dentists, especially for students and junior dentists.
Declaration of competing interest
The authors have no conflicts of interest relevant to this article.
Acknowledgments
This research was partially granted by the Special Research Project of the Southern Taiwan Science Park Precision Health Cluster of the National Science and Technology Council (BX-03-11-15-111).
Contributor Information
Guang Hong, Email: hong.guang.d6@tohoku.ac.jp.
Chun-Cheng Hung, Email: chuchh@kmu.edu.tw.
References
- 1.Clancy J.S., Lindquist T.J., Palik J.F., Lynn A., Johnson L.A. A Comparison of student performance in a simulation clinic and a traditional laboratory environment: three-Year Results. J Dent Educ. 2002;66:1331–1337. [PubMed] [Google Scholar]
- 2.Perry S., Bridges S.M., Burrow M.F. A review of the use of simulation in dental education. Simulat Healthc J Soc Med Simulat. 2015;10:31–37. doi: 10.1097/SIH.0000000000000059. [DOI] [PubMed] [Google Scholar]
- 3.Krishnan D.G., Keloth A.V., Ubedulla S. Pros and cons of simulation in medical education: a review. Education. 2017;3:84–87. [Google Scholar]
- 4.Coulter G.A., Grossen B. The effectiveness of in-class instructive feedback versus after-class instructive feedback for teachers learning direct instruction teaching behaviors. Eff Sch Pract. 1997;16:21–35. [Google Scholar]
- 5.Hajhamid B., Somogyi-Ganss E. Improving effectiveness of dental student's feedback and course evaluation. J Dent Educ. 2021;85:794–801. doi: 10.1002/jdd.12548. [DOI] [PubMed] [Google Scholar]
- 6.Epstein R.M. Medical education–assessment in medical education. N Engl J Med. 2007;356:387–396. doi: 10.1056/NEJMra054784. [DOI] [PubMed] [Google Scholar]
- 7.Burrows R.S. Understanding self-assessment in undergraduate dental education. Br Dent J. 2018;224:897–900. doi: 10.1038/sj.bdj.2018.437. [DOI] [PubMed] [Google Scholar]
- 8.Eva K.W., Cunnington J.P., Reiter H.I., Keane D.R., Norman G.R. How can I know what I don't know? Poor self-assessment in a well-defined domain. Adv Health Sci Educ Theory Pract. 2004;9:211–224. doi: 10.1023/B:AHSE.0000038209.65714.d4. [DOI] [PubMed] [Google Scholar]
- 9.Ward M., Gruppen L., Regher G. Research in self-assessment. Current state of the art. Adv Health Sci Educ. 2002;7:63–80. doi: 10.1023/a:1014585522084. [DOI] [PubMed] [Google Scholar]
- 10.Huang T.K., Yang C.H., Hsieh Y.H., Wang J.C., Hung C.C. Augmented reality (AR) and virtual reality (VR) are applied in dentistry. Kaohsiung J Med Sci. 2018;34:243–248. doi: 10.1016/j.kjms.2018.01.009. [DOI] [PubMed] [Google Scholar]
- 11.Schleyer T.K., Thyvalikakath T.P., Spallek H., Dziabiak M.P., Johnson L.A. From information technology to informatics: the information revolution in dental education. J Dent Educ. 2012;76:142–153. [PMC free article] [PubMed] [Google Scholar]
- 12.Jasinevicius T.R., Landers M., Nelson S., Urbankova A. An evaluation of two dental simulation systems: virtual reality versus contemporary non-computer-assisted. J Dent Educ. 2004;68:1151–1162. [PubMed] [Google Scholar]
- 13.Gottliet R., Lanning S.K., Gunsolley J.C., Buchanan J.A. Faculty impressions on dental students' performance with and without virtual reality simulation. J Dent Educ. 2011;75:1443–1451. [PubMed] [Google Scholar]
- 14.Abdalla R., Bishop S.S., Villasante-Tezanos A.G., Bertoli E. Comparison between students' self-assessment, and visual and digital assessment techniques in dental anatomy wax-up grading. J Dent Educ. 2020;25:524–535. doi: 10.1111/eje.12628. [DOI] [PubMed] [Google Scholar]
- 15.Berry M.C.C., Neto J.M.M., Souza M.I.C., Figueredo C.M.S., Reher V., Evans J.L. Effectiveness of technology-enhanced learning to improve periodontics educational outcomes: a systematic review. J Dent Educ. 2020;84:830–839. doi: 10.1002/jdd.12179. [DOI] [PubMed] [Google Scholar]
- 16.Li Y.N., Ye H.Q., Ye F., et al. The current situation and future prospects of simulators in Dental Education. J Med Internet Res. 2021;23 doi: 10.2196/23635. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17.Chang T.Y., Hong G., Paganelli C., et al. Innovation of dental education during COVID-19 pandemic. J Dent Sci. 2021;16:15–20. doi: 10.1016/j.jds.2020.07.011. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18.Nishioka T., Mayanagi G., Iwamatsu-Kobayashi Y., Hong G. Impact of the COVID-19 pandemic on dental clinical training and future prospects. J Dent Educ. 2021;(Suppl 3):1999–2001. doi: 10.1002/jdd.12562. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19.Chang T.Y., Hsu M.L., Kwon J.S., Kusdhany M.F.L.S., Hong G. Effect of online learning for dental education in Asia during the pandemic of COVID-19. J Dent Sci. 2021;16:1095–1101. doi: 10.1016/j.jds.2021.06.006. [DOI] [PMC free article] [PubMed] [Google Scholar]