Effect of computer-assisted-learning and simulation clinics on dental students’ cognitive and performance skills: panoramic image errors related to patient’s head position

Abstract

Objectives:

To assess dental students’ ability to recognize head positioning errors in panoramic (PAN) images after individual learning via computer-assisted-learning (CAL) and in a simulation clinic (SIM). Both cognitive skills and performance in patient examination were assessed.

Methods and materials:

60 students (mean age 23.25 years) participated in lectures on the relation between PAN-image errors and patient’s head position. Immediately after they took a test, based on which they were randomized to three groups: control (CON) group, CAL group, and SIM group (both CAL and training in a simulation clinic with a phantom). 4–5 weeks after intervention/no intervention, all students individually examined a patient with PAN-exposure. A blinded rater, not knowing group allocation, supervised patient exposure and assessed student’s performance (correct/incorrect head position in three planes). 1–2 weeks after, the students scored positioning errors in 40 PAN-images. Differences in cognitive test scores between groups were evaluated by ANOVA and in patient examination by χ² tests, and within-group differences by sign-tests.

Results:

No statistically significant difference in cognitive test scores was seen between the SIM and CAL group, while the CON group scored lower (p < 0.003). In all groups, several students positioned the patient incorrectly in the Frankfort horizontal plane. All students performed well in the sagittal plane. Students in SIM group positioned the patient more correctly in the coronal plane.

Conclusions:

Training with CAL increased students’ cognitive skills compared with a control group. Simulated patient exposure with a phantom increased to some extent their performance skills in examination of patients.

Introduction

Although computer technologies to distribute information have been available for many decades, outcome studies of e-learning or blended learning in the undergraduate dental radiology curriculum are still sparse. A recent review on studies on e-learning in oral radiology included 17 studies with quantitative outcome measures.¹ The authors suggested a classification of the studies as to whether the outcome measure was on students’ knowledge (cognitive field) or performance (“show how”). The outcome “knowledge” may moreover be assessed on different hierarchical levels as first suggested in the Bloom’s Taxonomy of cognitive skills,² which has recently been re-assessed.³ The lowest level in this hierarchy is “remember” and the highest is “analysis/assessment/synthesis”; i.e. in imaging research, this may often be the “diagnostic” level.

Six of the studies included in the review examined outcome on the analysis/diagnostic level,^4–9 and five of these showed a benefit for the student of the e-learning activity. Five studies related to students’ performance, while only three of these were “show how” studies, where a rater evaluated performance in full-mouth radiographic surveys and the bisecting angle technique, all assessed on manikins.^10–12 It was emphasized in the review that no study has assessed the performance level “show how” by the actual radiographic examination of a patient. In most of the previous studies, the cohort under investigation was “split” or two cohorts were followed, however, it was rarely stated whether or how the groups were randomized in a controlled (RCT) manner, and whether the raters were blind as to group belonging for the “show how” studies. An RCT-design is essential to obtain valid information from such studies.¹³

Further, none of the studies included in the review¹ dealt with panoramic radiography. Despite many advantages, it is stated in a worldwide used textbook¹⁴ that one of the disadvantages of panoramic imaging is the demand for a very accurate patient positioning to avoid errors and artifacts in the image. Image errors that originate in incorrect patient positioning in the panoramic unit are known to be related to the three planes: jaw position anterior or posterior to the image layer (coronal plane), flexion or extension of the patient’s head (Frankfort horizontal plane), and head/body rotation (sagittal plane). Further, the patient may rotate his/her head before exposure, even after correct positioning, known as a midline shift, which also gives rise to asymmetries in the size of the jaws and teeth in the image.¹⁴

The objectives of this randomized controlled trial were to assess dental students’ cognitive skills and their performance in a patient panoramic examination after participating in computer-assisted-learning (CAL) and simulation clinics. The outcome measures were knowledge on the analysis level, i.e. their ability to recognize panoramic image errors and relate these to patient’s head position, and performance in the clinic with a real patient exposure, i.e. to inform and position the patient correctly.

The hypothesis was that pre-graduate dental students’ cognitive and performance skills in panoramic image examination will increase after individual learning via a CAL program and practicing in a simulation clinic with a phantom.

Participants and methods

Dental students

60 dental students at the fifth semester (third year in dental school) participated in this randomized, controlled trial between September and November 2019. The students’ mean age was 23.25 years (range 20–38 years); 47 females and 13 males. The students were not aware from the beginning that they were to participate in this study, but were just told that radiology teaching sessions were mandatory.

Development of a computer-assisted-learning program of the relation between incorrect head positions and panoramic image errors

Four radiology teachers (LS, LHM, RSN, AW) developed a computer-assisted step-by-step guide (CAL program) using PowerPoint (Microsoft, USA). The program consisted of 40 slides, some with video sequences of, e.g. patient movement. A phantom (Dental Radiography Head Phantom - PH 47, Kyoto Kagaku, Kyoto, Japan) was used as the model for head positions in the panoramic unit (PlanmecaPromax, Planmeca Oy, Helsinki, Finland). First, an introduction to the panoramic principle showing drawings of the image layer and the correct position of the patient using the light indicators built into the panoramic unit was presented (Figure 1). Thereafter, head position errors were displayed in each plane, coronal, horizontal, and sagittal together with a systematic explanation and presentation of the position of the light indicators. Eventually, the subsequent panoramic image for that head position was displayed. The incorrect head positions are explained in Table 1, and the derived panoramic image for each position was displayed as seen in Figure 2.

Figure 1.

Correct head position of the phantom in the unit (a), drawing of image layer (b), and derived image of phantom (c).

Table 1.

Relation between error in head position/patient acting incorrectly and errors in the image

Relation between error in head position/patient acting incorrectly and errors in the image
1. Operator positioned the patient incorrectly in relation to the image layer, coronal plane
1.1. Anterior to image layer, canine light indicator too far posteriorly: horizontal reduction, narrow (unsharp) frontal teeth
1.2. Posterior to image layer, canine light indicator too far anteriorly: horizontal magnification, broad (unsharp) frontal teeth
2. Operator positioned the patient incorrectly in relation to flexion/extension of head, Frankfort horizontal plane
2.1. Chin tipped down/flexion, Frankfort horizontal light indicator not correct: ”too positive smile line”, large chin (sometimes chin missing in the image), sometimes spine projected over ramus in both sides, narrow, distorted frontal teeth
2.2. Chin tipped up/extension, Frankfort horizontal light indicator not correct: ”straight/negative smile line”, flat chin, overlapping posterior teeth
3. Operator positioned the patient incorrectly in relation to rotation of the head (no midline shift), sagittal plane
3.1. Rotation to the left, vertical light indicator in midsagittal plane between tooth 11 and 21): left side-posterior teeth enlarged compared to right side, overlap between premolars/molars in left side
3.2. Rotation to the right, vertical light indicator in midsagittal plane between tooth 11 and 21): right side-posterior teeth enlarged compared to left side, overlap between premolars/molars in right side
4. Operator positioned patient correctly, but patient turned his/her head before exposure (midline shift)
4.1. Turned head to the left, vertical light indicator shifted (to e.g. between tooth 12 and 11): right side-posterior teeth and lower jaw ramus enlarged compared to left side, no overlap or equal overlap in both sides
4.2. Turned head to the right, vertical light indicator shifted (to e.g. between tooth 21 and 22): left side-posterior teeth and lower jaw ramus enlarged compared to right side, no overlap or equal overlap in both sides

Figure 2.

Resulting images after head position errors in the phantom. The numbers refer to Table 1.

Theoretical lectures and pretest

All students participated in a lecture (2 × 45 min) on panoramic image characteristics and the relation between patient’s head position and image errors given by one of the radiology teachers (LHM). The panoramic unit and its light indicators for head positioning in the three planes was explained. As described in textbooks,¹⁴ four types of errors may be identified in a panoramic image, which are due to either the operator positioning the patient incorrectly, or the patient acting in some erroneous manner before/during exposure. Examples of both phantom and patient images after a correct head position and various head positioning errors were shown, and the relation between the specific image characteristics and the patient’s position in the panoramic unit was explained as shown in Table 1.

Two independent raters (radiology teachers: AW, LHM) had categorized 40 patient images according to error(s) in the image. If disagreement between the raters, the case was discussed, and a few cases were omitted and replaced with an image, where consensus was obtained. The correct status of the images was thereafter defined as: 5 images with no errors, 15 images with errors in positioning regarding the image layer, i.e. patient positioned too anteriorly or posteriorly in relation to the image layer; 12 images with errors regarding the Frankfort plane, i.e. head flexion or extension; 15 images with errors regarding head rotation, i.e. operator had positioned the patient’s head rotated to the left or right; and 3 images where patient had turned his head to one side (operator’s position was perfect, but patient had turned his head before exposure).

Immediately after the lecture, the students participated in a pretest using these 40 PAN-images of patients. The students had 1 h to complete the test. The students were to relate image characteristics to a correct or erroneous head position in the panoramic unit. For each image, they gave four scores, one for each of the head position planes – and patient’s head turn – using the scoring system described in Table 2. The students were given this description to guide their scores. Image quality with respect to contrast, sharpness, or density was not assessed in this test, nor was position of the patient’s tongue during the examination and presence of artifacts from earrings, etc.

Table 2.

Scores for each of three head position planes and patient’s head turn

No.	Plane	Indicator in unit	Position score
1	Coronal	Canine light	Correct Anterior Posterior
2	Horizontal	Frankfort horizontal light	Correct Chin tipped down Chin tipped up
3	Sagittal	Midsagittal light: no light shift	Correct Rotated left Rotated right
4	Sagittal, patient turned head	Midsagittal light: light shift	No Turned left Turned right

Control and intervention groups

10 students had not completed the assessment of all 40 cases, ranging from 1 to 18 missing images (i.e. number of missing scores ranged 4–72). Total number of correct scores was counted for each student, i.e. a correct score = agreement between student’s and raters’ answer. The distribution of the number of correct scores among all students was divided in thirds, best 1/3, middle 1/3, and worst 1/3. Based on their score counts, the students were randomly allocated to three groups within each third: a control (CON) group, a CAL group, and a patient simulation (SIM) group, who also learned by the CAL program (flow diagram in Figure 3). Thus, all three groups contained the same number of basically high-scoring, middle-scoring, and low-scoring students, i.e. 20 students in each group.

Figure 3.

Flowchart of the RCT. RCT, randomized controlled trial.

5–7 weeks after the lecture, the intervention started. The students were scheduled for radiology teaching during 8 working days, number of students per day ranged between 6 and 8. An introduction to the groups and group affiliation was given by an instructor (RSN).

CON group: The 20 students in the CON group (mean age 23.30, range 21–26) had no access to SIM training or the CAL program. This group had a seminar, where they were engaged in regular clinical activities in the radiology clinic.

CAL group: The 20 students in the CAL group (mean age 23.00, range 21–28) worked with the CAL program installed on two PCs in a quiet room at the radiology department for approximately 1 h. They worked two-by-two and discussed along the way. The teacher did not interfere. Thereafter, these students returned to their regular clinical activities in the radiology clinic.

SIM group: The 20 students in the SIM group (mean age 23.45, range 20–38) worked with both the CAL program and with hands-on training in the simulation clinic, with the head phantom (described above) simulating a patient in the PlanmecaPromax panoramic unit. The students worked two-by-two in the SIM clinic. They were instructed to first study the CAL program and then work with the phantom producing a total of nine panoramic exposures, placing the phantom correctly and in various incorrect positions. The nine positions can be read from Table 2: coronal plane (positions 1,2,3), horizontal plane (positions 2,3), sagittal plane (positions 2,3), and “patient” turned head (positions 2,3). One teacher (AW) supervised the students in the SIM clinic and demonstrated the characteristics of and how to work the panoramic unit; besides this, the students did not get any further help or information. After approximately 1.5 h, these students returned to their regular clinical activities in the radiology clinic.

Performance test: information to the patient and patient positioning

4–5 weeks after the intervention, the 60 students returned to the radiology clinic to perform a panoramic exposure of a patient. There were 6–8 students per day over 8 days. Each student individually examined a patient in the PlanmecaPromax panoramic unit. The patients were referred to the radiology department for panoramic imaging of the wisdom teeth or for orthodontic treatment planning.

Before this session, all students had received a written instruction on how to receive and instruct the patient before the examination. The students were also instructed in how to operate the panoramic unit before the patient was called. One rater (LHM), who had not taken part in the randomization of the students and therefore was blinded to group allocation, assessed students’ performance. The assessment included two items: the student’s performance in (1) informing the patient before and after exposure and (2) positioning the patient in the panoramic unit. After the rater’s assessment, in case patient positioning was incorrect, the rater corrected the position and explained why to the student before exposure.

The following five situations were scored for item (1) as a mini-cex (mini clinical evaluation exercise)¹⁵: the student’s call on the patient, information to the patient about the examination, removal of removable items from the face and mouth, information about tongue position in the palate, and information to the patient on the resulting image. There were three grades for each situation: 1 = below expectations; 2 = expected; 3 = above expectations. The student’s scores in the mini-cex were summed (max points achievable was 15 (3 × 5)). Item (2), positioning the patient, was scored as correct/incorrect for head position in the three planes as described in Table 2.

The rater’s assessments were subsequently deblinded according to student group belonging. Differences in scores between the groups CON, CAL, and SIM were tested with parametric statistics (ANOVA; followed by post-hoc tests) for the total score in the mini-cex. Differences between groups in correct/incorrect scores in head positioning were analyzed with binary tests for categorical data (χ² tests). Further, within each group and each positioning plane, the difference between number of correct and number of incorrect scores was tested by sign-tests.

Cognitive skills test: recognition of image errors and their relation to head position

1–2 weeks after the performance test (Figure 3), the students participated again in the cognitive test with the 40 PAN-images of patients with and without head positioning errors. The students had 1 h to complete the test. Number of correct answers was calculated for each student. The maximum number of scores that could be obtained was 160 (40 images x 4 error categories). Further, a detailed analysis was performed to evaluate the students’ ability to recognize each image error separately in the 40 images, i.e. how many times a specific head position error was correctly recognized in an image. Differences in test scores were compared between groups by parametric statistics (ANOVA followed by post-hoc tests). “SPSS” (IBM Corp., New York, NY; formerly SPSS Inc., Chicago, IL) was used for data evaluation, and the level of statistical significance was p < 0.05.

Results

Performance test: information to the patient and patient positioning

Table 3 shows the scores from the mini-cex regarding information to the patient in the three groups. The difference between the CON and CAL group was statistically significant (p = 0.035). No significant differences were found among these groups and SIM.

Table 3.

Students’ scores in the mini-cex for the CON, CAL, and SIM groups. The max score was 15

	CON	CAL	SIM
Mean score	10.85A	11.20B	11.10AB
SD scores	1.23	2.12	2.00
% score	72%	75%	74%

Different letters between groups indicate a statistically significant difference.

A vs B: p = 0.035.

Table 4 shows the scores for positioning a patient in the panoramic unit in three planes. For the overall score (all three planes combined), there were no significant differences between groups (p > 0.05). In the coronal plane, only students in the SIM group positioned significantly more patients correctly than incorrectly in relation to the image layer (sign-test, p < 0.01).

Table 4.

Scores for patient positioning in three planes in the performance test for students in the CON, CAL, and SIM groups

Plane	CON		CAL		SIM
	Correct	Incorrect	Correct	Incorrect	Correct	Incorrect
Coronal	15ns	5ns	14ns	6ns	18C	2D
Frankfort horizontal	8ns	12ns	13ns	7ns	10ns	10ns
Sagittal	17A	3B	17A	3B	19C	1D

ns = non-significant difference between number of correct and incorrect patient positioning scores within group.

Different letters within group indicate a statistically significant difference between number of correct and incorrect scores. A vs B: p = 0.01; C vs D: p < 0.01.

In the Frankfort horizontal plane, i.e. head flexion/extension, students in all groups had difficulties in positioning the patient correctly. No differences between number of correct and incorrect scores were seen within any group (sign-test, p > 0.05).

In the sagittal plane, students in all groups positioned significantly more patients correctly than incorrectly in relation to head rotation (sign-test within CON and CAL group, p = 0.01). In the SIM group, only one patient was incorrectly positioned in this plane (p < 0.01).

Cognitive skills test: recognition of image errors and their relation to head position

All students had completed the cognitive test, i.e. all 40 images were scored within the hour that was available. The students in the CAL and SIM groups obtained on average a higher number of correct scores than students in the CON group (p < 0.003). The number of correct scores was not significantly different between the CAL and SIM groups (Table 5).

Table 5.

Students’ scores in the cognitive test for the CON, CAL, and SIM groups

	CON	CAL	SIM
Mean score	93.1A	109.35B	105.35B
Range scores	72–120	88–132	82–124
% score	58.2%	68.3%	65.8%

Different letters between groups indicate a statistically significant difference.

A vs B: p < 0.003.

There were no significant differences between the CAL and SIM groups (p > 0.05) for any of the image errors tested separately, while number of correct findings for error type differed significantly between the CON and CAL groups (p < 0.04; except for item 1, error in the coronal plane (p = 0.22)), and between the CON and SIM groups (p < 0.026; except for item 2, error in the Frankfort horizontal plane (p = 0.09)).

Discussion

In this RCT, we tested various domain competences in dental students, randomly allocated to a control group and two intervention groups, a CAL group and a SIM group. The control group received the conventional teaching in the subject of panoramic image errors related to patient’s head position, as it has been conducted for several years in our dental radiology curriculum. It was anticipated that students, who learned by the CAL program, obtained higher scores in a cognitive test with the objective to recognize errors in panoramic images of patients and relate these to erroneous head positions, and that students, who moreover practiced in the SIM clinic (not e-learning), further increased their skills in performing a panoramic exposure in a patient. The hypotheses in general held true. It would have been informative to include one more group of students in the study design, a group with SIM only, thus one limitation of the study is that CAL and SIM cannot be compared separately. Two matters pointed against this design: (1) there were 60 students in this semester, and 4 groups would mean 15 in each group, providing a smaller sample size for subsequent statistical testing, and (2) the actual environment that was available for the study. The study had to run during those hours where the students were scheduled for radiology seminars, and only one panoramic unit was available during these hours. We decided to build the groups as add-ons, i.e. no intervention, CAL, and CAL + SIM.

Since there were 17% of the students, who had not completed the pretest immediately after the lectures and prior to the intervention, we refrained from performing a comparison between this test and the final cognitive test, which all students completed after the intervention. The students were unprepared for the trial when they arrived at the lecture on panoramic imaging, and some students may have had other obligations after the scheduled hour had passed. Other factors than low skills could therefore have influenced the fact that they did not complete the pretest, and a small bias in the allocation to groups may therefore be suspected.

More students in CAL and SIM groups recognized errors and their origin correctly in panoramic images; though statistically significant, the difference between these groups and controls was not extremely large (about 10%), and in the CAL and SIM groups, students achieved approximately 65% correct answers in the cognitive test. It must be born in mind that the structure of the learning program, CAL, and the sequence in the SIM clinic was chronological, i.e. first a certain head position in the panoramic unit was shown and thereafter the derived image, while in the cognitive test, it was vice versa, an image with/without errors was shown, and the student must deduce the erroneous head position from the image characteristics. This is however the clinical situation, where the dentist does not necessarily perform the patient examination, but must recognize possible errors in the image that should be corrected. In a previous study on the knowledge of graduating American dental students about panoramic imaging (not on e-learning), the mean percentage of correct answers for error and artifact recognition in panoramic images was 61% and thus on a line with results from our study.¹⁶

The performance test with a patient took place in a panoramic unit with the patient in the standing position. Only this particular unit was tested, which is another limitation of the study since the results cannot be directly transferred to units where the patient sits in a chair. Two positioning errors were significantly more often avoided in the SIM group than in the other two groups: positioning the patient anteriorly or posteriorly in relation to the image layer and head rotation. For head position in relation to the Frankfort horizontal plane, no group performed well. We speculate that the reason is that the phantom’s neck and height did not need adjustment between exposures in the simulation clinic – the phantom has a “stiff” neck, and the same “standing height” for every exposure. Further, no bite pin can be inserted into the phantom’s “mouth” – while in a patient, the chin rest, unit’s height, and the bite pin inserted between patient’s upper and lower incisors need to be set individually. Therefore, many of the students in all groups failed to recognize that the patient was standing with a curvilinear neck and flexed head. Educators should be aware of this in the future, when instructing students using the phantom for practicing patient exposure. The companies producing phantoms for radiology education should also consider this when developing new products, able to mimic all characteristics of a patient. Since it would be unethical to let the students expose the patient in case of an erroneous head position, the rater corrected those errors before exposure. A patient turning his/her head after positioning (light shift error in the sagittal plane) did not occur in this trial and was not assessed in the performance test.

Moreover, a mini-cex tested whether the student called and informed the patient correctly before and after the exposure. A mini-cex is a direct observation clinical evaluation tool, which is meant to give feedback to the student in situations, where the student interacts with patients in a clinical procedure.¹⁵ This clinical evaluation tool seems not to have been used much in dental educational research, but is meaningful for behavioral skills, which may not be easily qualified as correct/incorrect. There should be no differences among the groups for this task since this was trained neither in the CAL nor in the SIM group. In subsequent semesters, the students will receive further training, which may increase their skills in interpersonal communication.¹⁷

Few studies have assessed students’ skills in panoramic radiology. A web-based interactive program for learning panoramic anatomy was no more effective for dental hygienist students than lectures, and most students preferred class-room instruction.¹⁸ Our RCT on panoramic imaging seems to be the first study, in which students’ performance was evaluated. Similar to other RCTs reported in oral radiology for assessing spatial information and image quality in radiographs,^19,20 students in our study trained between 1 and 1.5 hours with the CAL or the CAL + SIM intervention. Our interpretation is that the time spent with the CAL program is beneficial to the students in terms of cost-effectiveness, at least when evaluated within this short period after the intervention. It may be speculated if training with the phantom is also cost-effective, since the costs for such phantoms are still quite high in opposition to costs for CAL development.

In conclusion, in this RCT, students improved their cognitive skills after working with a CAL program compared with a control group. Students who worked with simulated patient exposure on a phantom, performed somewhat better in a panoramic examination of a patient than those who did not train with a phantom.