Abstract
Introduction
Teaching dental caries removal is limited by the material and methods available in the preclinical teaching space. Plastic teeth do not simulate the tactile feel of a lesion and natural teeth do not allow for standardised training and assessment. A novel method for simulating caries has been reported. Here, to investigate the construct validity of a caries simulation, whether haptic simulation could contribute to the understanding of caries removal, the performance of first-year dental students on the haptic simulation exercise is compared with that of experienced dentists.
Method
A virtual block comprising healthy dentine, pulp, enamel and a carious lesion with significant spread along the amelodentinal junction (ADJ) was developed for the Simodont dental trainer. The case was presented to 112 first-year students and 17 clinicians following a 15 min training period on a block which contained green caries and displayed live progress throughout the exercise. All participants were given the same verbal instructions: to remove all unsupported enamel and caries along the ADJ while retaining as much healthy tissue as possible.
Results
Clinicians performed better than the dental novices in precision and overall performance. Clinicians removed more material on average, except for healthy dentine, of which similar amounts were removed by both groups.
Discussion
We presented a novel haptic caries exercise and investigated the construct validity of the task. The simulation may bridge the gap between preclinical and clinical dental education in caries removal.
Conclusion
Clinically experienced dentists outperformed novices on a haptic caries simulation exercise. The exercise may be a useful tool for assessing conceptual understanding of caries removal.
Keywords: simulation, simulation-based learning, simulation-based education, virtual reality, validation
Key messages.
What is already known on this subject
Dental students are normally taught caries removal using plastic, or extracted, teeth. Haptic simulation may offer a way to simulate the tactile aspects of caries removal in a way not currently available to students.
There is evidence to suggest the construct validity of haptic training as a manual dexterity training tool. However, there is limited work investigating the use of the haptics for more clinically orientated tasks such as caries removal.
What this study adds
The haptic exercises investigated require skills more likely to be present in clinically experienced individuals than novice dental students.
The exercises investigated may be a valid tool for identifying students which require additional support in grasping the core concepts related to caries removal.
Introduction
Caries removal is a fundamental skill for any dentist, as there is a high prevalence of caries in the UK.1 2 Operative treatment relies primarily on the ability to distinguish between healthy and carious tissue, and the carious lesion, using the tactile response of hand instruments or the dental handpiece. The procedure is further complicated by changing trends in the amount of the carious lesion which should be removed, as there is a growing emphasis on retaining some of the softer, but uninfected tissue (affected dentine), while ensuring a clean preparation around the amelodentinal junction (ADJ).3
Today, dental students in the UK are taught cavity preparation using a combination of plastic and extracted teeth. Most plastic teeth do not contain any pathology or anatomical features, resulting in the cariology task being reduced to a shape cutting exercise rather than a pathologically and clinically driven exercise. This manual dexterity facet of dental education has traditionally been relied on heavily for training and assessment, but evidence indicates that the ability to neatly cut a plastic tooth does not map well to predicting future clinical ability.4 5 Plastic teeth containing pathology and anatomical features are available but are currently considered too costly by most institutions for regular educational use. Problematically, plastic teeth do not allow for any insight into tactile perception, despite this being recognised as an essential requirement for successful caries removal.6 Further, even the more elaborately constructed plastic teeth reportedly fail to realistically, and uniformly, simulate the difference in tactile feedback by the various simulated tissues.7
Extracted teeth are often used as an adjunct to remedy the shortcomings of plastic. However, this provides a challenge in that the pathology is, in many cases, so extreme that extraction was deemed the most appropriate treatment. Furthermore, each tooth is anatomically and pathologically unique which complicates the standardisation of the learning experience and assessment. Finally, the use of any physical teeth (biological or plastic) requires the presence of a dental tutor and laboratory facilities. These are an expensive and time-limited resource, yet it is generally understood that almost all students would be capable of achieving the necessary skill levels, given sufficient practice time.8 9 With this in mind, the use of haptic methods to complement traditional training may offer an opportunity to simulate aspects of caries removal that are difficult to reproduce in the phantom head. Automatic performance scoring and feedback along with the lack of need for constant staff presence (due to the low-risk environment) means that haptic simulation may also offer students the chance to practice as much as they need in their own time to build their confidence and hone their skills to the required level. The automatic scoring system may also preclude any discrepancies in a student’s perceived performance, and performance as assessed by a tutor, as investigated by San Diego.10
Bakker et al 11 found that practising virtual shape cutting exercises on a simulator resulted in a similar improvement in ability as that of someone practising using traditional shape cutting exercises. Mirghani et al 12 found statistical differences in performance between novice dental students and fifth-year students when tasked to drill a shape in a virtual block simulating a preclinical manual dexterity exercise. These studies indicate a level of construct validity (that a test measures what it reports to be measuring) in the haptic system and that it is possible to measure improvement in manual dexterity, if not clinical ability, using the haptic simulator.
A small number of studies have been undertaken to compare the performance of dental novices to dental professionals using haptic caries exercises, including Eve et al.13 This exercise contained a green carious lesion of a uniform density. The lesion did not spread along the ADJ, undermining the enamel, and as such demanded little in terms of comprehension of correct caries removal. Despite this, the study did find that clinicians removed more of the lesion, more efficiently, than novice users. Yamaguchi14 presented a caries exercise which contained unsupported enamel, but in this case the carious lesion was coloured black, which is likely to affect caries removal decisions. Findings indicated improved performance with practice, but did not include comparison across cohorts with differing amounts of experience. Ria et al 15 reported, as part of the hapTEL research into haptic simulation in dentistry, a trend in less iatrogenic damage being undertaken during caries removal in virtual teeth by participants who spent more time within the exercise before starting initial drilling. The hapTEL experiments included extensive introduction to the theories surrounding correct caries removal; the relation between time spent prior to excavation and reduced iatrogenic damage may suggest that this haptic system scenario encouraged student reflection, and as such, learning. However, the carious teeth available in the hapTEL system do not include a tactile gradient and thus differ from the exercises presented herein.
In summary, there is evidence to suggest the construct validity of haptic training as a manual dexterity training tool.11 12 However, there is limited work investigating the use of the haptics for more clinically orientated tasks, such as caries removal and cavity design. In an attempt to remedy this, a novel method has recently been developed to generate haptic caries for the Simodont.16 The exercises simulate the organic growth of pathology through a virtual block: penetrating through the enamel and spreading out along the ADJ before burrowing toward the dental pulp. Each material within the block simulates the appropriate tactile feedback, calibrated by three clinicians, through the handpiece when interacted with. The lesion gradually ranges from very soft to the hardness of affected dentine, and the cases are automatically and objectively marked on completion.
Crucially, the colour of the carious lesion can be controlled. In some training exercises, the lesion can be obviously different in colour, while in others it looks no different to healthy tooth (as is sometimes the case clinically). Thus, the exercises allow students to practice caries removal based solely on tactile feedback or based on a combination of visual cues (discolouration) and tactile feedback. To our knowledge, the non-discoloured caries scenario has not previously been simulated, despite being a real and challenging clinical occurrence.
This paper aims to investigate the validity of haptic caries exercises, where visual cues on discolouration are not present. We denote ‘validity’ to mean that the exercise requires a set of skills which are clinically relevant, and thereby, are a potential tool which could benefit cariology training. To assess this, we compare the results achieved by 112 first-year students with those obtained by 17 clinical practitioners.
Method
One hundred and eleven first-year dental students and 17 clinical practitioners participated in the study. The test subjects were anonymised within each group. Data collection was undertaken in the same week, approximately halfway through the first academic year. At this stage, students had attended an introductory lecture on dental caries, but had yet to be introduced to the preclinical skills laboratory, and, in most cases, had no experience using the dental handpieces. All clinicians participating were clinically active teaching staff. Each user was given a prewritten verbal introduction outlining the goal of the exercise: to remove all caries at the ADJ, without excessive removal of the healthy, surrounding tissue. The participants had access to a virtual slow speed and high speed bur. A duration of 15 min was given for each exercise.
Two caries blocks were generated: one introduction block containing green caries and a test block without any discolouration. Both lesions were created by using unique, and arbitrary, shapes as a ‘seed images’ from which the caries grew (figure 1). See Ref. 16 for further details on creating the haptic caries simulation. The introduction block had the user’s current progress displayed live on screen. The user was able to restart or repeat the exercise freely within the set time of 15 min. The colour and live score during the introduction gave the user the opportunity to ensure they understood the task. The time limit allowed for one generous attempt or several repeated attempts. The test block did not contain any colour other than that of enamel and dentine, and no progress information was displayed to the user. This meant that the task was executed entirely based on the haptic feedback from the handpiece and design decisions made based on the haptic feedback, the preliminary verbal introduction and, in the case of clinicians, previous experience. One attempt was allowed, with 15 min being the maximum time allowed. Results were then exported and analysed in Excel.
Figure 1.
From top left, ‘seed image’ and top amelodentinal junction (ADJ) slice of the introduction block; ‘seed image’ and top ADJ slice of test block. Second row: volumetric view of introduction block (left) and test block (right). Bottom row: Labelled orthographic slice view of the test block.
Participant precision score was calculated using the number of voxels drilled in each of the segments: enamel, dentine, ADJ caries (including any unsupported enamel), (deep) caries and pulp. In theory, a user removing all the segment representing unsupported enamel and ADJ caries (equivalent to ~7% of the complete block) would obtain a perfect score. Any additional material removed, with the exception of deep caries would penalise the score, resulting in:
Precision score = % of ADJ caries removed – (% of enamel+% of dentine+% of pulp).
Two-sample t-tests, assuming equal variances, were employed for all comparisons between students and clinicians (see table 1). Variances between the two groups were not statistically different using Levene’s test.
Table 1.
Participant scores and significance between the two cohorts
| Student score, mean (%) (SD) |
Clinician score, mean (%) (SD) |
Difference (%) (95% CI) |
P value | |
| Precision score | 19.69 (27.38) | 39.34 (34.1) | −19.65 (−13.9 to 12.2) | 0.009 |
| ADJ caries | 53.60 (21.32) | 82.82 (17.13) | −29.21 (−38.73 to 19.7) | <0.001 |
| Deep caries | 59.63 (26.39) | 70.77 (18.47) | −11.13 (−21.65 to 0.62) | 0.029 |
| Enamel | 4.76 (5.94) | 8.29 (6.40) | −3.53 (−6.96 to to 0.10) | 0.025 |
| Dentine | 2.38 (2.53) | 2.47 (2.60) | −0.09 (−1.49 to to 1.31) | 0.892 |
ADJ, amelodentinal junction.
Results
All 112 participants of the student group were enrolled first-year students. All 17 clinicians were clinical teaching staff.
Clinicians were significantly more precise than students in removing caries without excessively removing the non-carious parts of the block (figure 2). The difference in precision score between student and clinicians was −19.65% (95% CI −13.9% to −12.2%), p=0.009.
Figure 2.
Precision score by the two groups. A perfect score could be achieved by removing all 8492 ‘amelodentinal junction (ADJ) caries’ voxels without any other damage. The score was calculated by counting the number of ADJ caries voxels removed and subtracting any healthy (dentine and enamel) voxels. Error bars represent 95% CI. The difference in precision score between students (n=112) and clinicians (n=17) was −19.65% (95% CI −13.9% to −12.2%). Students removed a mean percentage of 19.69%±27.38%, clinicians 39.34%±34.1%.
Clinicians removed significantly more caries at the ADJ (ADJ caries) than students (figure 3A). The difference was −29.21% (95% CI −38.79% to 19.7%), p<0.001.
Figure 3.
(A) Amelodentinal junction (ADJ) caries removed. (B) Deep caries removed. (C) Amount of enamel voxels removed. (D) Amount of unaffected dentine removed. Error bars represent 95% CI. Clinicians removed significantly more caries at the ADJ (ADJ caries) than students (students (n=112): 53.60%±21.32%; clinicians (n=17): 82.82%±17.13%) (figure 3A). The difference was −29.21% (95% CI −38.79% to 19.7%). Students removed 59.63%±26.3%, clinicians 70.77%±18.47% of deep caries (figure 3B). The difference in deep caries removed was −11.13% (95% CI −21.65% to −0.62%).
Fourteen (82%) clinicians and 69 (62%) student participants removed more than half of the deep caries available, despite this not being a prerequisite to the task (figure 3B). The difference in deep caries removed was −11.13% (95% CI −21.65% to −0.62%), p=0.029.
Please refer table 1 for the complete results.
No participants exposed the pulp.
Discussion
We present a novel haptic simulation which aims to introduce dental students to the tactile and cognitive aspects of caries removal, with no visual cues based on discolouration; these traits are not currently possible to simulate within the traditional preclinical teaching space.
Clinicians achieved a significantly better mean precision score than first-year participants, indicating that the caries block exercise requires a skill set potentially more likely to be present in a clinician than novice dentist. It is noteworthy that a number of underlying causes may affect task performance (denoted as ‘precision’ in this study), including fine motor control, understanding of the criteria, understanding of the biological processes involved in caries growth, clinical experience and more. That clinicians achieved significantly better precision scores may indicate that the haptic caries exercises require one, or more, of these underlying traits.
The clinician cohort removed significantly more enamel than the student group. Enamel undermined by the lesion was included in the ‘ADJ caries’ segment, while healthy enamel commenced just beyond the extent of the lesion. Therefore, to detect the margin of the cavity, it is likely that a certain amount of healthy enamel needed to be removed around the periphery of the lesion. If no enamel voxels were removed, this would indicate an inadequate clearing of the cavity margin. Our results may suggest that clinicians showed a better ability to detect the cavity margin and design the appropriate cavity preparation accordingly (figure 3C, D). The wide spread in precision scores by the clinician cohort may also be an artefact of the precision score penalising any removal of enamel and highlights an issue introduced when generating an overall ‘precision’ score. This issue may also be reflected in the unexpectedly low mean percentage precision score by clinicians (39.34%) despite the noticeably greater amount of ADJ caries removed by clinicians compared with the student cohort.
As for the large variance in some of the scores, this could reflect clinical experience at odds with the task set during the experiment—the task was to remove the carious lesion, but communicated as ‘remove all softer tissue below the enamel’; while a simple task to the students, this may come across as conflicting messages to a clinician who may have been trained to remove beyond the carious lesion. With this in mind, deep caries was not considered in the precision score. 82% of clinicians and 62% of student participants removed more than half of the deep caries present in the block (figure 3B). While clearing the ADJ is standard procedure for practitioners, the amount of deep caries to remove is not.3 6 Hence, we may expect a greater deviation among participants with the deep caries removal than the ADJ caries. Deep caries did not have any impact on the precision score in this study.
Perhaps uniquely to these haptic exercises, the segments within the exercise blocks were not mapped to hardness, but rather to clinical relevance. As such, the success rate of the participant would rely not only on tactile feedback but also on the ability to understand the verbal instructions given prior to the task.
The cases were automatically and objectively marked to exclude the possibility of bias within the scoring. As discussed, the appropriate amount of deeper caries to remove from a lesion is a good example of a scenario where even a blinded examiner would be likely to be subjective on appropriate execution of the task. By relying on automatic scoring and not taking deep caries removal into account when calculating the precision score, we do not suffer the risk of validating our caries simulation to a specific ‘school of thought’ within caries removal.
In the case of deep caries, students and clinicians removed similar amounts on average indicating less individual certainty about how much deep caries to remove. This uncertainty could be due to lack of experience (students) or lack of a clinically led ideal (staff), both of which would result in the same pattern of deep caries removal.
No information was collected about participants other than cohort designation. The lack of information collected for each participant is a limitation of this study: The amount of time spent on the haptic simulator previously could have an impact on case performance, but this was not recorded. However, as clinicians on average outperformed the novice group, with the student cohort being more likely to have had any previous experience using the simulators (due to outreach events, etc), this appears unlikely to be a factor in this study. Likewise, there is a chance that some of the student participants would have had previous experience using dental instruments prior to starting dental school, but this does not appear likely to affect findings, due to the large sample size. Further, information on the level of overall clinical experience among the clinician cohort, and an estimate of how often they were likely to perform caries tissue removal may have provided insights into the spread of performance scores generated by the clinician cohort, and the lack of such information is thus a likely limitation of the current study.
This study set out to investigate the construct validity of the newly developed simulated caries removal exercise. The significant difference in precision scores indicates a level of construct validity. Other valuable methods of testing the task, including test–retest reliability, have not been investigated at this stage. Retesting a group of participants could have provided a valuable insight into whether practising the task may improve the participant’s overall performance. Seeing whether the difference between the two cohorts would remain following practice would be of interest. The haptic exercise could thus benefit from further investigation.
Due to the challenge of recruiting clinicians, the number of students compared with staff participants was imbalanced (17 staff: 112 students). To ameliorate the impact of this imbalance on our interpretations of the analysis, for each dependent variable we performed a Levene’s test for equality of variance. The homogeneity of variance assumption across the two groups was not violated for any of these measures, thus indicating that the independent t-tests applied here were appropriate for these data.
Lastly, we may echo the sentiment stated by Eve et al (2014) that participants are enthusiastic about training on exercises that feel more clinically relevant than much of what is currently available during the preclinical undergraduate course. Future work includes adapting the caries growing algorithm to create haptic carious teeth.
Conclusion
A haptic caries exercise has been tested on dental novices and dental clinicians. Results identify a reliable difference in precision between the two groups. Clinicians perform better at the task, removing more caries and incurring less iatrogenic damage than the dental novices.
This study has validated the ability of the simulated caries removal exercise to distinguish between novices and experienced clinicians. As such, the simulation may be a useful tool to identify which students require additional support in understanding the correct approach to caries removal, a core concept in preclinical education.
Acknowledgments
Thanks to Esther Buur at Moog for technical support during case development.
Footnotes
Twitter: @faisalmushtaq
Contributors: CO and AK contributed to conception and design of study, acquisition, analysis and write-up. AD and PF contributed to conception and design of study, acquisition and write-up. JW contributed to analysis and write-up. FM contributed to concept and design of study, analysis and write-up.
Funding: FM is supported by a Research Grant from the EPSRC (EP/R031193/1) and a fellowship from the Alan Turing Institute.
Competing interests: None declared.
Ethics approval: The study was approved by the University of Leeds Dental Research Ethics Committee.
Provenance and peer review: Not commissioned; externally peer reviewed.
Data availability statement: Data are available on request from the corresponding author CO.
References
- 1. White D, Pitts N, Steele J, et al. Disease and related disorders – a report from the adult dental health survey. Heal Soc Care Inf Cent 2011:1–55. [Google Scholar]
- 2. Schwendicke F, Dörfer CE, Schlattmann P, et al. Socioeconomic inequality and caries: a systematic review and meta-analysis. J Dent Res 2015;94:10–18. 10.1177/0022034514557546 [DOI] [PubMed] [Google Scholar]
- 3. Banerjee A, Frencken JE, Schwendicke F, et al. Contemporary operative caries management: consensus recommendations on minimally invasive caries removal. Br Dent J 2017;223:215–22. 10.1038/sj.bdj.2017.672 [DOI] [PubMed] [Google Scholar]
- 4. Curtis DA, Lind SL, Brear S, et al. The correlation of student performance in preclinical and clinical prosthodontic assessments. J Dent Educ 2007;71:365–72. [PubMed] [Google Scholar]
- 5. Fugill M. Defining the purpose of phantom head. Eur J Dent Educ 2013;17:e1–4. 10.1111/eje.12008 [DOI] [PubMed] [Google Scholar]
- 6. Nascimento MM, Behar-Horenstein LS, Feng X, et al. Exploring how U.S. dental schools teach removal of carious tissues during cavity preparations. J Dent Educ 2017;81:5–13. [PubMed] [Google Scholar]
- 7. Delgado AJ, Walter R, Behar-Horenstein LS, et al. Are all Dentiform teeth with simulated caries the same? A six-year retrospective study in preclinical operative dentistry. J Dent Educ 2015;79:1330–8. [PubMed] [Google Scholar]
- 8. Ericsson KA, Krampe RT, Tesch-Römer C. The role of deliberate practice in the acquisition of expert performance. Psychol Rev 1993;100:363–406. 10.1037/0033-295X.100.3.363 [DOI] [Google Scholar]
- 9. Ericsson KA. Deliberate practice and the acquisition and maintenance of expert performance in medicine and related domains. Acad Med 2004;79:S70–81. 10.1097/00001888-200410001-00022 [DOI] [PubMed] [Google Scholar]
- 10. San Diego JP, Newton T, Quinn BFA, et al. Levels of agreement between student and staff assessments of clinical skills in performing cavity preparation in artificial teeth. Eur J Dent Educ 2014;18:58–64. 10.1111/eje.12059 [DOI] [PubMed] [Google Scholar]
- 11. Bakker D, Lagerweij M, Wesselink P, et al. Transfer of manual dexterity skills acquired in the Simodont, a dental haptic trainer with a virtual environment, to reality: a pilot study. Bio-Algorithms and Med-Systems 2010;6:21–4. [Google Scholar]
- 12. Mirghani I, Mushtaq F, Allsop MJ, et al. Capturing differences in dental training using a virtual reality simulator. Eur J Dent Educ 2018;22:67–71. 10.1111/eje.12245 [DOI] [PubMed] [Google Scholar]
- 13. Eve EJ, Koo S, Alshihri AA, et al. Performance of dental students versus prosthodontics residents on a 3D immersive haptic simulator. J Dent Educ 2014;78:630–7. [PubMed] [Google Scholar]
- 14. Yamaguchi S, Yoshida Y, Noborio H, et al. The usefulness of a haptic virtual reality simulator with repetitive training to teach caries removal and periodontal pocket probing skills. Dent Mater J 2013;32:847–52. 10.4012/dmj.2013-174 [DOI] [PubMed] [Google Scholar]
- 15. Ria S, Cox MJ, Quinn BF, et al. A scoring system for assessing learning progression of dental students' clinical skills using haptic virtual Workstations. J Dent Educ 2018;82:277–85. 10.21815/JDE.018.028 [DOI] [PubMed] [Google Scholar]
- 16. Osnes C, Keeling A. Developing haptic caries simulation for dental education. JSS 2017;4:29–34. 10.1102/2051-7726.2017.0006 [DOI] [Google Scholar]