Problem-based learning in continuing medical education

Abstract

Objective

To investigate the effects of problem-based learning (PBL) in continuing medical education.

Data sources

PubMed, MEDLINE, EMBASE, CINAHL, and ERIC databases were searched for randomized controlled trials published in English from January 2001 to May 2011 using key words problem-based learning, practice-based, self-directed, learner-centered, and active learning, combined with continuing medical education, continuing professional development, post professional, postgraduate, and adult learning.

Study selection

Randomized controlled trials that described the effects of PBL on knowledge enhancement, performance improvement, participants’ satisfaction, or patients’ health outcomes were selected for analysis.

Synthesis

Fifteen studies were included in this review: 4 involved postgraduate trainee doctors, 10 involved practising physicians, and 1 had both groups. Online learning was used in 7 studies. Among postgraduate trainees PBL showed no significant differences in knowledge gain compared with lectures or non–case-based learning. In continuing education, PBL showed no significant difference in knowledge gain when compared with other methods. Several studies did not provide an educational intervention for the control group. Physician performance improvement showed an upward trend in groups participating in PBL, but no significant differences were noted in health outcomes.

Conclusion

Online PBL is a useful method of delivering continuing medical education. There is limited evidence that PBL in continuing education would enhance physicians’ performance or improve health outcomes.

Problem-based learning (PBL) was developed about 50 years ago to teach medical students basic sciences in the clinical context. It is recognized as a successful innovative learning method in undergraduate medical education.¹ Problem-based learning was defined as “an instructional (and curricular) learner-centered approach that empowered learners to conduct research, integrate theory and practice, and apply knowledge and skills to develop a viable solution to a defined problem.”² Long-term effects of PBL on undergraduate education include positive changes in physicians’ competency after graduation, emphasizing the validity of PBL in enhancing practice performance.³

The attractiveness of PBL as a learner-centred, interactive educational approach has prompted many to adopt this method in postgraduate and continuing medical education (CME), even though the evidence for its effectiveness in CME is lacking.^2,4,5 Previous systematic reviews have not been able to prove that PBL is superior in terms of knowledge gain to traditional methods of teaching, such as lectures, in higher medical education. Further, physicians who learned through lectures performed better in examinations.^6,7 On the other hand, physicians preferred PBL and considered it a challenging and enjoyable learning method.⁷

A systematic review on the effectiveness of PBL in CME that included all randomized controlled trials on PBL in CME published from 1974 to 2000 found limited evidence of the effectiveness of PBL in increasing knowledge, enhancing performance, or improving health outcomes.⁷ This study found moderate evidence that medical practitioners were more satisfied with the PBL method.⁷

Several advances are being made toward innovation in postgraduate education and CME, and many countries worldwide are engaged in examining and updating their educational strategies; PBL involving tutor-facilitated, problem-based learning sessions, usually conducted in small groups, is an attractive concept for continuing education.

Problem-based learning can be a resource-intensive method of delivering CME on a regular basis for institutions or organizations that offer CME. The CME landscape has changed considerably over the past 10 years, and the last review of the effectiveness of PBL in CME was done more than a decade ago. There have also been concerns that PBL has been oversold, and its value in continuing education to change physician behaviour and patient outcomes has been questioned.⁵ As such, there is a knowledge gap related to the effectiveness of PBL in CME.

The objective of this review was to investigate the usefulness of PBL in improving knowledge, performance, and satisfaction of medical practitioners who participate in CME.

Methods

Data sources

PubMed, MEDLINE, EMBASE, CINAHL, and ERIC databases were searched for studies published between January 2001 and May 2011. The key words problem-based learning, practice-based, self-directed, learner-centered, and active learning were used for the search. The search results were combined with those of another search performed using the key words continuing medical education, continuing professional development, post professional, postgraduate, and adult learning. The search was limited to randomized controlled trials published in English.

Study selection

Studies that examined PBL were scanned and categorized according to their reported outcomes; studies that described the effect of PBL on knowledge enhancement, performance improvement, participants’ satisfaction, or patients’ health outcomes were selected.

The selected studies were analyzed using 5 main quality criteria: randomization, follow-up, blindness, intention to treat, and group similarity at the start of the study.⁷ Each criterion was scored on a scale of 1 to 10, with a maximum total score of 50. Studies that scored 25 or more were considered to be of high quality, and those that scored less than 25 were considered to be of low quality, as was the case in the previous review in 2002.⁷ Both authors independently assessed the quality of each selected study.

The studies were categorized based on whether the participants were postgraduate trainees, practising physicians, or both, and each study was reviewed to extract the identified outcome variables (participants’ knowledge, performance, satisfaction, and patients’ health outcomes). The effect of each variable was categorized as positive, negative, or no effect. Traditional learning methods or no intervention were used as the control in many studies.

RESULTS

A total of 15 randomized controlled studies were included in this review (Table 1).^8–22 All the studies involved case-based or active learning, and the participants comprised postgraduate trainees in 4 studies,^8–11 practising physicians and postgraduate trainees in 1 study,¹² and practising physicians in the remaining 10 studies.^13–22 Of these, 14 studies (93%) were of high quality (Table 2),^8–22 as compared with only 2 of the 6 studies (33%) included in the previous review conducted in 2002 with the same quality analysis criteria.⁷ Out of 15 studies reviewed, 7 used case-based e-learning or online teaching as their PBL method^{9,15–17,19,20,22}; 1 study used a moderator and described online group discussions.²²

Table 1.

Description of included studies

STUDY	COUNTRY	PARTICIPANTS					OUTCOME VARIABLES
			EDUCATIONAL INTERVENTION		NO. OF PARTICIPANTS		PARTICIPANT KNOWLEDGE	PARTICIPANT PERFORMANCE	PARTICIPANT SATISFACTION	HEALTH OUTCOME
			PBL	CONTROL	PBL	CONTROL
Carrero et al,8 2007	Spain	First-year residents	1-h group discussion on preanesthetic assessment	1-h lecture-based session on same topic	25	29	Improved in both with little difference	NA	NA	NA
Cook et al,9 2009	United States	First-, second-, and third-year residents	Online case-based modules on osteoporosis, cervical cancer screening, tobacco dependence, and dementia	Online non–case-based teaching of same subjects	65	65	Improved in both with little difference	NA	Higher with PBL format	NA
Haidet et al,10 2004	United States	Residents	1-h small group discussions within a large group setting on effective use of diagnostic testing	1-h didactic lecture on same topic	36	27	Both groups had improved knowledge gain and retention with no significant difference between the groups	NA	Lower perception of the educational value of the PBL session	NA
Smits et al,11 2003	Netherlands	Residents	1 d/wk for 4 wk of small group learning sessions on mental health problems	1 d/wk of lectures for 4 wk on same content	59	59	Improved in both groups with no significant differences	Increased in both groups but significantly more in the PBL group (P < .05)	Less satisfied with the PBL method	NA
Searle et al,12 2002	Australia	Gynecologists, registrars, and visiting medical officers	Interactive workshop on dysfunctional uterine bleeding	Reading materials	18	28	NA	Increase in evidence-based behaviour in the intervention group	NA	No change
White et al,13 2004	Canada	Family physicians	1-h small group session on asthma management	1-h lecture on same topic by the same facilitator	23	29	Increase in knowledge gain and retention with both with no significant differences	NA	More positive satisfaction with PBL	NA
Taylor et al,14 2004	United Kingdom	Physicians, administrators, and allied health workers	Half-day small group workshop on critical appraisal skills	None	73	72	Positive	No effect	NA	NA
Hugenholtz et al,15 2008	Netherlands	Occupational physicians	30-min online module on mental health	30-min lecture based on same topic	37	35	Improved in both with no significant difference	NA	NA	NA
Short et al,16 2006	United States	Primary care physicians	4-h asynchronous online multimedia sessions on intimate partner violence	None	23	29	Positive	Positive	Positive	NA
Harris et al,17 2002	United States	Primary care physicians, emergency physicians, and orthopedic surgeons	2-h online program on domestic violence	None	28	37	Positive	No effect	Positive	NA
Hugenholtz et al,18 2008	Netherlands	Occupational physicians	3 half-days over 2 wk of small group learning with 10 PBL sessions on evidence-based medicine	None	49	59	Improved	Positive	Perceived as valuable; no improvement in perceived self-efficacy	NA
Allison et al,19 2005	United States	Primary care physicians	4 multimodal online modules on chlamydia screening	Online flat text education on women’s health	96	95	NA	Positive; decline in chlamydia screening tests was less in the intervention group	NA	NA
Curtis et al,20 2007	United States	Primary care physicians, rheumatologists	3 online interactive modules on glucocorticoid-induced osteoporosis	Online text-based teaching on chronic illness other than osteoporosis	78	75	NA	No statistically significant difference although the intervention group conducted more screening tests and interventions	NA	No significant difference
Harris et al,21 2005	Canada	Primary care physicians	8, 1-h teleconference and online diabetes education sessions	None	43	47	NA	Effect positive in some aspects	Positive	Improved categorization of patients with diabetes but no change in final outcome of reducing HbA1c
Stewart et al,22 2005	Canada	Family physicians	2 case-based online learning modules each lasting 2 wks on type 2 diabetes and preventive care in family practice	None	27	31	Positive	Positive	NA	NA

HbA_1c—hemoglobin A_1c, NA—not assessed, PBL—problem-based learning.

Table 2.

Quality of studies: Each criterion was scored on a scale of 1 to 10, with a maximum total score of 50; studies that scored 25 or more were considered to be of high quality.

STUDY	RANDOMIZED	FOLLOW-UP	INTENTION TO TREAT	BLINDED	GROUPS SIMILAR AT START	TOTAL SCORE	STUDY QUALITY
Searle et al,12 2002	10	8	10	0	10	38	High
Harris et al,17 2002	10	3	10	0	10	33	High
Smits et al,11 2003	10	8	10	0	10	38	High
Haidet et al,10 2004	10	3	0	10	10	33	High
Taylor et al,14 2004	10	8	10	0	10	38	High
White et al,13 2004	10	8	10	0	10	38	High
Allison et al,19 2005	10	8	10	0	10	38	High
Stewart et al,22 2005	10	5	10	0	5	30	High
Harris et al,21 2005	10	8	10	0	10	38	High
Short et al,16 2006	10	8	10	0	10	38	High
Carrero et al,8 2007	10	3	10	0	10	33	High
Curtis et al,20 2007	10	5	10	0	10	35	High
Hugenholtz et al,15 2008	10	5	10	10	10	45	High
Hugenholtz et al,18 2008	10	8	10	0	10	38	High
Cook et al,9 2009	10	3	0	0	10	23	Low

Postgraduate education

Four studies examined PBL compared with didactic learning in postgraduate medical education using pretest-posttest assessments and showed similar increases in knowledge gain or knowledge application with both methods of learning. These teaching sessions included clinical skills such as preanesthetic assessment,⁸ specific disease management and preventive health,⁹ clinical reasoning in effective use of diagnostic testing,¹⁰ and management of mental health problems.¹¹

One of the studies incorporated principles of PBL in large group settings. Students worked in small groups within the larger group for 30 minutes and then had some content delivery during discussion; this was compared with a traditional 60-minute lecture to make the best use of tutors’ time and to increase the number of learners. This study found that learners’ perceptions of their engagement, the value of the session, and whether learning objectives were met were higher in the PBL group.¹⁰ In another study, students preferred the case-based format of online PBL compared with online learning without patient scenarios.⁹ It should be noted that all the resources for learning the content were provided as Web-based links in this study.

Students perceived PBL to have less educational value than a lecture-based format when content was covered by engaging students in small group sessions 1 day a week for 4 weeks.¹¹ This study used self-reported performance indicators to measure performance in practice and found no significant difference between the 2 groups.

An Australian study randomized specialist and trainee physicians providing gynecology services in 6 public hospitals to receive either no intervention or to attend a PBL workshop on hysteroscopies and dilation and curettage in women younger than 40 years of age with dysfunctional uterine bleeding. They evaluated performance using a questionnaire with clinical scenarios and practice audits.¹² This study found that practice change did not occur when the actual practice was audited, although evidence-based behaviour change was present in the intervention group with the survey.¹²

Continuing education

Three studies focused on the effectiveness of PBL in knowledge enhancement for practising physicians: 2 studies randomized physicians to a PBL intervention group or a control group (didactic lectures) to examine knowledge improvement in asthma management¹³ and critical appraisal skills.¹⁴ Knowledge uptake regarding asthma management was similar in both groups; the subjective assessment of the educational value of the session was higher in the PBL group compared with the lecture-based group (4.36 vs 3.93, P = .04).¹³ The critical appraisal teaching study analyzed the cost of introducing small group workshops and concluded that educators should consider cost-effectiveness when planning educational interventions.¹⁴ The PBL group had a slightly higher knowledge score in critical appraisal skills training but there was no difference in participants’ attitude toward evidence or their ability to critically appraise literature when both groups were compared.¹⁴ When delivered as short online case-based learning, PBL for mental health education produced knowledge improvement similar to that of lecture-based learning.¹⁵

Two studies showed that physicians’ knowledge, and self-reported confidence and self-efficacy in managing domestic violence improved significantly (P < .05) after PBL delivered as Internet-based self-study, compared with controls with no intervention.^16,17 Another study compared PBL delivered as 3 half-day sessions over 2 weeks with facilitated case method learning with no intervention. The authors found improvement in self-reported professional performance in the intervention group compared with the controls, but no difference in self-efficacy or job satisfaction was noted between groups.¹⁸

Effect of PBL on physicians’ performance and health outcomes

A multicomponent online PBL program to increase chlamydia screening showed that screening rates were higher in the intervention group compared with the control group with no intervention.¹⁹ Online PBL on glucocorticoid-induced osteoporosis management combined with performance audit and feedback showed no significant effect on the quality of osteoporosis care, although physicians who completed all the PBL modules showed a trend toward higher rates of screening and prescribing treatment for osteoporosis.²⁰

Problem-based learning on diabetes education delivered by teleconferencing, compared with a control group with no intervention, did not improve overall patient glycemic levels but led to improvement in categorizing patients with diabetes to the correct glycemic categories and increased the number of patients treated with insulin.²¹ Another study examining PBL delivered online to improve preventive care and diabetes management employed chart audits and standardized patients to evaluate the effects of the PBL and compared outcomes with those of a control group with no intervention. The intervention group showed no significant differences in practice with the standardized patients but showed improvement in knowledge and improvement of patient care for diabetes management according to chart audits.²²

Discussion

The original definition and the mode of delivery of PBL for postgraduate and continuing education seems to have evolved to accommodate the growing needs for distance education by using modern technology such as online learning. As such, there was considerable expert or teacher involvement in building the modules and providing the resources. The objective of this study was not to assess online PBL. However, online PBL emerged as a popular method of delivering CME and it needs to be evaluated further, as it could be very useful for physicians in remote areas.

Nearly all the studies involving residents indicated less positive perceptions of the value of PBL, which is not unexpected, as the control groups almost always consisted of another method of education, often with face-to-face contact with an expert; most of the studies examining CME did not have expert lectures for the comparison group. This factor should also be taken into consideration when comparing perceptions of postgraduate trainees with those of practising physicians.

One of the studies describes active PBL in large group settings where students worked in small groups within the large group setting for 30 minutes and then had some content delivery during discussion.¹⁰ This interactive method of incorporating principles of PBL in large group settings might be a good way to maximize tutors’ time and increase the number of learners in CME.

Online learning seems to be becoming more popular and it is an important option for future CME providers. However, considerable time commitment might be required to develop, maintain, and update online modules. Pooling resources and sharing online modules among institutions and professional bodies might be a useful strategy for the optimum use of available information. Barriers that would need to be addressed include the financial costs of maintaining and updating modules and intellectual property tied to building PBL modules.

Previous studies have concluded that PBL can positively enhance clinical reasoning.^23–25 However, they indicated that the improvement was in backward reasoning rather than forward reasoning.^24,25 A study conducted among undergraduate medical students showed that diagnostic accuracy was higher in the group who had backward reasoning.²⁶ There is a lack of studies examining the effectiveness of backward reasoning in CME, and assessment of the effectiveness among practising physicians is limited. Examination of clinical reasoning among practising physicians and the effects of PBL are potential areas of study for future research.

Previous reviews have demonstrated that PBL could effectively enhance competencies and clinical performance.^26,27 Social and cognitive competencies such as dealing with uncertainty, recognizing ethical issues related to health care, communication skills, and self-directed learning are more likely to be improved by PBL methods.^3,27 A cohort study in undergraduate medical education comparing PBL with lecture-based learning found no significant difference in clinical competencies after graduation.²⁸ We found no studies demonstrating improvement in clinical competencies with PBL in CME.

Although participants’ satisfaction with PBL was graded as high in the review conducted in 2002,⁷ our findings showed that satisfaction was moderate and more participants were satisfied with online PBL methods. There was dissatisfaction with online PBL among residents, as participants perceived that PBL had a low educational value and did not meet learning objectives.^10,11 The new learning environment, the possible discomfort working with new peers, and reduced contact with the teacher might have led to low perceived value of PBL among residents.

A cross-sectional study conducted among physicians attending a CME activity indicated that although most participants recognized the effectiveness of interactive, case-based methods to retain information and change practice, most preferred lecture-based teaching.²⁹ Other studies showed that online CME was more favourable than small group interactive PBL.^30,31 These studies concluded that Internet-based CME is substantially similar to or superior to interactive group CME with regard to knowledge gain.^30,31 This point should be considered when planning CME activities in PBL formats.

Few studies measured health outcomes, an important measure of the effectiveness of teaching methods.³² More research is needed to examine the long-term effects of PBL, as physicians’ behaviour change and health outcomes are influenced by a multitude of other factors such as the strength of evidence for guidelines, system factors, targeting behaviour, expected health outcomes, and the measurements used to evaluate these outcomes.

Limitations

Only randomized controlled trials that investigated the effect of PBL on CME over the past 10 years were included, and other relevant studies might have been excluded. Resident education studies were included, as they were the only available studies to look at certain outcomes such as knowledge acquisition with PBL; however, the motivations and constraints are different for postgraduate education and CME. The effect sizes, if reported, were small and were often not analyzed or reported.

The assessment tool to evaluate the quality of selected studies was chosen because it was used in the previous review on the topic. As with the previous review, this review also showed that many studies did not have comparable educational interventions for control groups, and many control groups had no educational interventions. As such, their assessment would have represented an evaluation of the education as well as the mode of delivery and cannot be assumed to be an evaluation of only the PBL methodology. The reliability and validity of the assessment tools were not described well in several studies, and confounding factors for the outcomes were not discussed.

Conclusion

Online PBL is perceived to be an effective educational strategy by physicians. There is limited evidence that PBL in continuing education enhances physicians’ performance or improves health outcomes. Thus, it is recommended that educators consider all factors, including cost effectiveness and the role in distance learning when implementing PBL methodology in continuing education.

EDITOR’S KEY POINTS

• Problem-based learning (PBL) is comparable to lectures with regard to knowledge improvement in postgraduate and continuing medical education, and there is limited evidence that PBL in continuing education enhances physicians’ performance or improves health outcomes.

• Online PBL is perceived to be an effective educational strategy by physicians. Educators need to consider all factors, including cost effectiveness, when implementing PBL methodology in continuing education.

POINTS DE REPÈRE DU RÉDACTEUR

• L’utilisation de l’apprentissage par problème (APP) pour la formation médicale continue et postdoctorale est aussi efficace que les cours traditionnels pour améliorer les connaissances; toutefois, il y a peu d’indication que son utilisation améliore la performance des médecins ou la santé des patients.

• Il semble que l’APP en ligne soit une stratégie éducationnelle efficace pour le médecin. Les enseignants devraient prendre en compte tous les facteurs, incluant le rapport coût-bénéfice, avant de recourir à l’APP en formation médicale continue.