Follow-up assessment of problem-based learning in dental alveolar surgery education: a pilot trial

Introduction

The objectives of problem-based learning (PBL) in medical education include the acquisition of a rich body of knowledge that is integrated across a wide variety of disciplines, the development of clinical problem-solving abilities and team-working skills, and the development of lifelong learning habits and abilities1., 2.. PBL is student-centred, consists of small-group work and is independent and self-directed, providing a motivating and enjoyable approach to medical education3., 4.. PBL improves the academic results of medical students5., 6.. Moreover, PBL teaching may provide dental graduates with enhanced abilities to communicate and cooperate with others7., 8..

Most of the dental school curriculum is well integrated across various disciplines7., 8., 9., 10., 11., 12.. To some extent, the study of oral and maxillofacial surgery is a postgraduate field or area of professional training. Accordingly, few reports have addressed the implementation of oral and maxillofacial surgery-integrated PBL courses in dental undergraduate training. However, dental alveolar surgery, an essential component of oral and maxillofacial surgery, is as important as other dentistry specialties (e.g. endodontics or operative dentistry) in the training of dental undergraduates. Several disciplines teach diagnostic and treatment planning, relevant to dental alveolar surgery, using the traditional dental education approach. Accordingly, the integration of diagnostic and treatment planning in the dental curriculum may help dental undergraduates meet clinical preparedness requirements. Literature searches reveal limited information on the effects of an integrated dental alveolar surgery PBL course for dental undergraduates and its relationship to their performance after graduation13., 14..

Therefore, we designed a PBL course featuring integrated dental alveolar surgery. The course was introduced to one class of students in their fourth year of study during the autumn semester, in 2007, as a randomised controlled trial. We performed a follow-up survey 3 years after completion of the integrated education programme to evaluate the outcomes of the PBL course on dental alveolar surgery. We compared the graduates in the PBL and lecture-based groups with respect to their clinical problem-solving and professional collaboration abilities at the end of the course as well as 3 years after graduation.

Methods

This study was performed with fourth-year undergraduate students in 2007 who were studying dental alveolar surgery at a Chinese dental school. Consistent with the teaching programme of oral and maxillofacial surgery, 72 hours were dedicated to the dental alveolar surgery course. All students signed written informed consent forms to participate in the study, which was approved by the Institutional Ethics Committee of the School of Somatology, China Medical University. The research was performed in accordance with the World Medical Association's Declaration of Helsinki.

Ninety students who agreed to participate in the study were randomly divided into two groups – the PBL group (n = 42) and the traditional lecture-based teaching (TLBT) group (n = 48) – using a random number table. None of the students had been exposed to PBL before this project or to the topic of dental alveolar surgery. The students in both groups were comparable with respect to age and gender. Comparability was also tested by comparing their third year of academic performance. They were grouped such that the previous academic performance of the students in each group was similar. The PBL group comprised seven age- and gender-matched subgroups, with each group consisting of six students. Each subgroup was assigned a tutor who had completed the standardised China Medical University PBL teacher education programme and was given five clinical problems that were designed to present complex situations related to dental alveolar surgery. The set-up of the PBL programme is shown in Tables 1 and 2. The TLBT group was taught using the traditional lecture-based course. The number of PBL hours versus the number of didactic lecture hours is shown in Table 3. The overall learning objective for both groups was to acquire preparedness for clinical practice associated with dental alveolar surgery.

Table 1.

The set-up of problem-based learning (PBL) cases and learning objectives

	Brief description of cases	Learning objectives
1	Adult patient with progressive periodontitis	The indication of tooth extraction and patient evaluation
2	Upper premolar apical periodontitis with the maxillary sinus extended over the apex	Assessment of the risk of oroantral communication during extraction Treatment planning for odontogenic maxillary sinusitis The management of oroantral communication
3	Mandibular cystic lesions associated with impacted third molar	Indication for third molar removal Presurgical clinical and radiological assessment for third molar Risk assessment
4	Teenager with an unerupted maxillary canine	Treatment planning for impacted canines and premolars Orthodontic considerations
5	Aged medically compromised patient with hyperplasia and a shallow vestibular sulcus	Principles of preprosthetic surgery for edentulous patients Management of medically compromised patients Indication for vestibuloplasty

Table 2.

Time allocation for each case

Activity	Hours
Overview of case and to define learning objectives	0.5
Students’ self-directed learning and discussion (Subgroup)	4
To give further clinical information and some questions (Each subgroup differently)	0.5
Students’ self-directed learning and discussion (Subgroup)	4
Students presenting case and answering questions (PBL group)	1
PBL group summary lecture	1

PBL, problem-based learning.

Table 3.

Time allocated to learning dental alveolar surgery in traditional lecture-based teaching (TLBT) and problem-based learning (PBL) groups

Study point	TLBT (72 hours)	PBL (72 hours)
Introduction of dental alveolar surgery	2	2
Didactic lectures	62	0
Training for PBL	0	2
Problem/case-based discussion	0	60
Clinical observation	4	4
Summary lecture	2	2
Review and feedback	2	2

At the end of the course, all students were required to take a theoretical examination and complete a clinical case analysis and a brief questionnaire. The examinations were scored as percentages. A senior oral surgeon, who was blinded to the student group assignments, rated the clinical case analysis of each student on a scale of 0–10 (in which 0 is the worst and 10 is the best). Finally, the students evaluated their interests in learning and assessed their motivation to cooperate with their classmates on a scale of 0–10, where 0 indicated a completely negative response (lack of interest and lack of motivation) and 10 indicated a completely positive response (plenty of interest and plenty of motivation). A score of 5 indicated the midpoint or a neutral response.

The supervisors were the dentists who supervised the graduates during their first 3 years of dental practice. Three years after completion of the PBL course, follow-up study was performed via a telephone interview and questionnaire. The questionnaire consisted of quantitative items and open-ended questions. The quantitative items were measured on a scale of 0–10, with 0 indicating a completely negative response (lack of preparation, negative attitude, lack of motivation and weak ability) and 10 indicating a completely positive response (full preparation, positive attitude, plenty of motivation and strong ability). The first section of the quantitative questionnaire evaluated the students’ self-perceived preparedness for clinical practice with respect to their professional knowledge of dental alveolar surgery. The second section asked the respondents to rate their attitudes and behaviours concerning lifelong learning. The third section asked the respondents to evaluate their abilities and willingness to collaborate with other colleagues over the past 3 years. Each section included the three domains shown in Table 4. The overall score of each section was determined by averaging the scores of the three domains for each section. The open-ended question for the PBL group was related to how the PBL teaching programme helped the students develop lifelong learning and collaboration abilities. The respondents in the TLBT group were asked, ‘What did you think was most important for developing lifelong learning and collaboration abilities?’. Furthermore, all respondents were required to provide their opinions on the weaknesses of the teaching programme. The questionnaire for the supervisors of the graduates contained only quantitative items. These rating forms were similar in content and format to those completed by the graduates, with the exception of the item, ‘Feel helpful in making treatment plan decisions through discussion’ in the third section, which was replaced with the ‘Teamwork abilities’ measure.

Table 4.

Survey measures of preparedness for clinical practice, lifelong learning behaviours and attitudes, and collaborative willingness and abilities (the results presented are those obtained 3 years after graduation)

Variable	Self-ratings			Supervisors’ ratings
	TLBT (n = 37)	PBL (n = 32)	P value	TLBT (n = 35)	PBL (n = 31)	P value
Preparedness for clinical practice related to dental alveolar surgery
Manage medically compromised patients	6.65 ± 1.48	7.19 ± 1.33	>0.05	6.74 ± 1.29	7.13 ± 1.18	>0.05
Develop treatment planning when tooth extraction should be indicated	6.41 ± 1.32	7.38 ± 1.31	0.010*	6.74 ± 1.12	7.48 ± 1.39	0.090
Manage the impacted tooth	6.92 ± 1.36	6.88 ± 1.41	>0.05	6.69 ± 1.47	6.84 ± 1.27	>0.05
Overall assessments of clinical preparedness	6.66 ± 0.83	7.15 ± 0.81	0.044*	6.72 ± 0.62	7.15 ± 0.82	0.113
Lifelong learning attitudes and behaviours
Be interested in updated technology and professional knowledge	6.14 ± 1.23	7.25 ± 1.39	<0.001*	6.31 ± 0.93	7.19 ± 1.25	0.016*
Apply evidence-based medicine in clinical practice	6.87 ± 1.08	6.66 ± 0.94	>0.05	6.86 ± 1.14	6.58 ± 1.12	>0.05
Critically evaluate professional literature	6.27 ± 1.10	6.56 ± 1.24	>0.05	6.40 ± 1.06	6.55 ± 1.03	>0.05
Overall assessments of lifelong learning	6.42 ± 0.63	6.60 ± 0.70	>0.05	6.52 ± 0.60	6.77 ± 0.61	>0.05
Collaborative willingness and abilities
Feel helpful to reach treatment plan decision through discussion	5.08 ± 1.61	6.53 ± 1.30	<0.001*
Teamwork abilities				4.74 ± 1.54	6.00 ± 1.07	0.002*
Consult with other physicians frequently	5.35 ± 1.59	6.78 ± 1.24	<0.001*	5.37 ± 1.61	6.52 ± 1.21	0.007*
Communicate effectively with colleagues	5.32 ± 1.47	5.28 ± 0.96	>0.05	4.97 ± 1.20	5.42 ± 1.31	>0.05
Overall assessments of collaboration	5.25 ± 0.88	6.20 ± 0.61	<0.001*	5.03 ± 0.82	5.98 ± 0.76	<0.001*

^*Denotes significant differences at P < 0.05.

Quantitative items were measured on a scale of 0–10, in which 0 indicated a completely negative response (lack of preparation, negative attitude, lack of motivation and weak ability) and 10 indicated a completely positive response (full preparation, positive attitude, plenty of motivation and strong ability). A score of 5 indicated a midpoint or a neutral response.

PBL, problem-based learning; TLBT, traditional lecture-based teaching.

All responses for the quantitative items were assessed on the previously described scale of 0–10. Two investigators evaluated the answers to the open-ended question. Consensus was acquired after discussion. Similar comments were sorted thematically. The frequency of the comments was given as a percentage of the total comments.

Finally, the data were analysed using the Statistical Package for the Social Sciences (Windows version 16.0; SPSS Inc., Chicago, IL, USA). The examination and case analysis scores, ratings of study interests and ratings of the collaborative motivations at the end of the course were compared using t-tests. The chi-square test was used to compare the response percentages. The follow-up data were analysed using one-way analysis of variance (ANOVA) followed by Tukey's post hoc test. Values of P < 0.05% were considered significant.

Results

Part A: data collected at the end of the course

The examination grades and the case-analysis scores of the students are shown in Table 5. No significant between-group differences in the mean grades and case-analysis scores were observed. The PBL students reported a greater level of study interest and motivation to collaborate with their classmates (P < 0.001; Table 6).

Table 5.

Distribution of examination marks and scores of case analysis between the groups (the results presented are those obtained at the end of the course)

Variable	TLBT (n = 48)	PBL (n = 42)	P
Examination marks	78.02 ± 9.01	77.62 ± 10.14	0.843
Case analysis scores	6.75 ± 1.39	7.12 ± 1.27	0.195

PBL, problem-based learning; TLBT, traditional lecture-based teaching.

Table 6.

Students’ self-assessment of their study interests and motivations of collaboration at the end of the course

Variable	TLBT (n = 48)	PBL (n = 42)	P
Study interests	6.25 ± 1.16	7.14 ± 1.30*	<0.001
Collaboration motivations	5.56 ± 1.20	7.14 ± 1.07*	<0.001

^*Denotes significant differences at P < 0.05.

Values were scored on a scale of 0–10, in which 0 indicated a completely negative response (lack of interest and lack of motivation) and 10 indicated a completely positive response (plenty of interest and plenty of motivation).

PBL, problem-based learning; TLBT, traditional lecture-based teaching.

Part B: data collected 3 years after graduation

In total, 69 of the 90 graduates returned their questionnaires, and 66 of the 90 supervisors returned their questionnaires. No significant differences were observed regarding the response percentages of the graduates or the supervisors in the PBL and TLBT groups (Table 7).

Table 7.

Percentage responses of graduates and supervisors

Variable	TLBT (n = 48)		PBL (n = 42)		P
	n	%	n	%
Graduates	37	77.08	32	76.19	0.979
Supervisors	35	72.92	31	73.81

PBL, problem-based learning; TLBT, traditional lecture-based teaching.

The PBL group graduates rated their preparedness for practicing clinical dentistry as 7.15 on a 10-point scale. This score was significantly higher than that in the TLBT group (6.66; P = 0.044). Graduates in the PBL group rated themselves higher than those in the TLBT group regarding their ability to develop a treatment plan and to determine whether tooth extraction is indicated. The respondents provided similar ratings on their preparedness to manage medically compromised patients and the management of impacted teeth. There were no statistical significant differences regarding the supervisor assessments, although the preparedness of the PBL group was rated higher than that of the TLBT group.

No significant between-group differences were observed in terms of the overall assessment of the graduates’ behaviours or their attitudes toward lifelong learning, according to both the graduate and supervisor ratings shown in Table 4. The two groups were equally likely to apply evidence-based medicine in clinical dentistry and critically evaluate and review professional information from textbooks, journals and electronic resources. Graduates from the PBL group provided higher self-ratings than did graduates from the TLBT group regarding interest in updating technology and professional knowledge.

Graduates of the PBL group rated their overall willingness to collaborate and their abilities higher than did graduates of the TLBT group. This result was consistent with their supervisors’ evaluations. The respondents in the PBL group felt better able to reach a treatment plan decision through discussion and were more likely to consult with professionals and other health-care staff than were students in the TLBT group. The supervisors rated the graduates in the PBL group higher than they rated the graduates in the TLBT group regarding their teamwork abilities and willingness to collaborate. No significant between-groups differences were observed regarding their ability to communicate effectively with their colleagues.

The graduate and supervisor results were similar regarding the overall assessments of collaboration.

More than half (56%) respondents in the PBL group expressed the feeling that PBL provided them with the opportunity to develop self-directed lifelong learning and communication skills. Several (22%) of the respondents mentioned that the weakness of PBL was its lack of guidance in learning. Of all graduates of the TLBT group, 65% believed that the inclusion of case discussion in PBL may be an appropriate method for fostering collaborative abilities in undergraduates. The predominant weakness (51%) listed by the TLBT group was the lack of integrated professional knowledge; the second most frequent answer (16%) was the insufficient encouragement for students to improve their problem-solving and critical thinking abilities.

Discussion

We designed an integrated PBL teaching intervention programme for dental alveolar surgery. This randomised controlled study evaluated the outcomes of this teaching programme. We compared graduates of PBL and TLBT programmes in terms of their clinical problem-solving and professional collaboration abilities at the end of the course, as well as 3 years after graduation. In the present study, the academic results did not differ between the PBL and TLBT groups. After 3 years, however, a follow-up evaluation of the graduates and their supervisors showed that the PBL graduates reported better preparedness for practicing clinical dentistry in the area of dental alveolar surgery than did the TLBT students. Furthermore, most of the graduates believed that the case discussion included in the PBL process was the proper method to help students develop collaborative willingness and abilities, as well as communication skills, during their undergraduate education.

The academic outcomes at the end of the course showed that the students in both groups achieved the learning objectives for dental alveolar surgery. In addition, no significant between-group differences were observed. According to the self-assessments, however, after graduation, the students in the PBL group reported a greater level of study interest and motivation for collaboration. This finding may be explained by the fact that incorporation of active learning is shown to help students improve their interest in learning2., 8..

Both the supervisor ratings and self-ratings of both of the groups were higher than 5 on a scale of 0–10. These findings indicate that all graduates were prepared for the clinical practice of dental alveolar surgery.

PBL courses emphasise early clinical exposure2., 15., 16.. Preclinical learning is integrated with multidisciplinary and appropriate clinical problems. The PBL students were more deeply impacted by the potential clinical problem scenarios than the TLBT students. In contrast, the TLBT students received the dental courses from several departments with little integration, which may have left gaps in their professional knowledge and allowed them to feel that their professional knowledge was incomplete. When faced with clinical practice, the students had to integrate their professional knowledge learned previously. This finding may explain why the graduates of the TLBT group perceived themselves as being less prepared than the PBL graduates.

The major aims of the undergraduate dental alveolar surgery course are to provide students with the ability to practice clinical dentistry with professional knowledge and oral surgery skills. Specifically, the students should be competent in diagnosing and managing third molar-related problems. They are expected to understand the indications and contraindications of exodontia and preprosthodontic surgery. With this knowledge and with the goal of improving our dental education, we administered a survey to evaluate our graduates regarding their preparedness for clinical dentistry in dental alveolar surgery, regardless of the major in which they enrolled after graduation.

According to their responses, the students could recall, 3 years after graduation, what they had been taught regarding dental alveolar surgery. We hypothesised that teaching and learning styles may have a greater effect on clinical performance within the first 3 years of practice than in subsequent years. Thus, a subsequent evaluation was performed 3 years later. Most of the graduates from the School of Stomatology at China Medical University were employed at a public hospital, which was also consistent with the results of this survey. Thus, the working experience of the respondents may have had a small effect on their responses, as employees of public hospitals have more opportunities for lifelong learning in China.

Graduates from the PBL group valued communication skills and felt helpful when communicating effectively with professionals and colleagues. The between-group differences were consistent with their assessment at the end of the course. Graduates from both groups believed that group discussion could also help students develop their ability to communicate as well as to develop their organisational and interpersonal skills. The responses to the open-ended questions revealed that most of the graduates were positive about integrated problem-based education. The supervisors gave a higher rating for the teamwork abilities than did the graduates. Thus, these findings indicate that PBL may improve the students’ communication competencies. These results were consistent with the findings of other studies2., 9.. The benefits of improving communicative willingness in PBL may result from the greater opportunities for discussion, presentation and active learning.

The students who participated in this study tended to be deeply affected by traditional Chinese culture. According to the doctrines of Confucianism, ‘Courtesy, Loyalty, Benevolence, and Credit’ are often required for communication between people. Chinese students are traditionally hesitant to communicate with others voluntarily¹⁷. Furthermore, the traditional examination-driven teaching system creates a learning environment in which students learn professional knowledge in a passive manner. Students rarely ask questions in class and never object to the topics being taught¹². The feedback of the open-ended questionnaire also revealed that these dental students were not used to communicating with their classmates about their professional knowledge and that they lacked the opportunity to develop their abilities via a traditional education. Zhang et al.⁸ reported that Chinese students who received PBL education had a greater inclination towards teamwork, which is consistent with our finding that the PBL graduates had a higher collaborative willingness than their counterparts. The PBL teaching and learning system encourages communication and discussion among group members, resulting in the willingness of students to communicate and collaborate. Therefore, the findings of this study support the notion that PBL improves students’ desire to communicate and collaborate with each other, especially for Chinese students.

This study did not reveal significant differences between the graduates’ behaviours or their attitudes toward lifelong learning, which contrasted with the results of a study of the long-term effects of PBL at McMaster University¹¹. One possible explanation for this finding is that the students have their own learning habits before their fourth year of undergraduate education. It is difficult to change students’ attitudes and behaviours after only half a semester of a new programme or course.

The consistency between the self- and supervisor ratings was high for both groups across behaviours and habits. The consistency of the results further supports the conclusions drawn from our study.

A limitation of our study is that only one course of PBL in one semester at one school was analysed, limiting the generalisation of the study results to all dental students.

With this limitation in mind, we conclude that PBL with the integrated dental alveolar surgery teaching programme may improve students’ preparedness for clinical practice and help undergraduates develop the desire to collaborate.

Acknowledgements

The authors would like to thank the survey respondents that made this research possible. The research was supported by Teaching Reform Fund of China Medical University (YDJK2015018) and Teaching Research Fund of School of Stomatology, China Medical University (J101593-15-02).

Conflict of interest

There were no competing interests.

Author contributions

X.B. conceived and execution the study and drafted the article. X.Z. performed collection of the data and statistical analysis. X.W. participated in the design of the study. L.L. led on conception of the study and participated in its design and coordination. Q.L. helped to draft and revise the manuscript. All authors have approved the final version of the article submitted.