Continuing education meetings and workshops: effects on professional practice and healthcare outcomes

Louise Forsetlund; Mary Ann O'Brien; Lisa Forsén; Leah Mwai; Liv Merete Reinar; Mbah P Okwen; Tanya Horsley; Christopher J Rose

doi:10.1002/14651858.CD003030.pub3

. 2021 Sep 15;2021(9):CD003030. doi: 10.1002/14651858.CD003030.pub3

Continuing education meetings and workshops: effects on professional practice and healthcare outcomes

Louise Forsetlund ^1,^✉, Mary Ann O'Brien ², Lisa Forsén ¹, Leah Mwai ³, Liv Merete Reinar ¹, Mbah P Okwen ⁴, Tanya Horsley ⁵, Christopher J Rose ¹

Editor: Cochrane Effective Practice and Organisation of Care Group

PMCID: PMC8441047 PMID: 34523128

Abstract

Background

Educational meetings are used widely by health personnel to provide continuing medical education and to promote implementation of innovations or translate new knowledge to change practice within healthcare systems. Previous reviews have concluded that educational meetings can result in small changes in behaviour, but that effects vary considerably. Investigations into which characteristics of educational meetings might lead to greater impact have yielded varying results, and factors that might explain heterogeneity in effects remain unclear. This is the second update of this Cochrane Review.

Objectives

• To assess the effects of educational meetings on professional practice and healthcare outcomes

• To investigate factors that might explain the heterogeneity of these effects

Search methods

We searched CENTRAL, MEDLINE, Embase, ERIC, Science Citation Index Expanded (ISI Web of Knowledge), and Social Sciences Citation Index (last search in November 2016).

Selection criteria

We sought randomised trials examining the effects of educational meetings on professional practice and patient outcomes.

Data collection and analysis

Two review authors independently extracted data and assessed risk of bias. One review author assessed the certainty of evidence (GRADE) and discussed with a second review author. We included studies in the primary analysis that reported baseline data and that we judged to be at low or unclear risk of bias. For each comparison of dichotomous outcomes, we measured treatment effect as risk difference adjusted for baseline compliance. We expressed adjusted risk difference values as percentages, and we noted that values greater than zero favour educational meetings. For continuous outcomes, we measured treatment effect as per cent change relative to the control group mean post test, adjusted for baseline performance; we expressed values as percentages and noted that values greater than zero favour educational meetings. We report means and 95% confidence intervals (CIs) and, when appropriate, medians and interquartile ranges to facilitate comparisons to previous versions of this review. We analysed professional and patient outcomes separately and analysed 22 variables that were hypothesised a priori to explain heterogeneity. We explored heterogeneity by using univariate meta‐regression and by inspecting violin plots.

Main results

We included 215 studies involving more than 28,167 health professionals, including 142 new studies for this update.

Educational meetings as the single intervention or the main component of a multi‐faceted intervention compared with no intervention

• Probably slightly improve compliance with desired practice when compared with no intervention (65 comparisons, 7868 health professionals for dichotomous outcomes (adjusted risk difference 6.79%, 95% CI 6.62% to 6.97%; median 4.00%; interquartile range 0.29% to 13.00%); 28 comparisons, 2577 health professionals for continuous outcomes (adjusted relative percentage change 44.36%, 95% CI 41.98% to 46.75%; median 20.00%; interquartile range 6.00% to 65.00%))

• Probably slightly improve patient outcomes compared with no intervention (15 comparisons, 2530 health professionals for dichotomous outcomes (adjusted risk difference 3.30%, 95% CI 3.10% to 3.51%; median 0.10%; interquartile range 0.00% to 4.00%); 28 comparisons, 2294 health professionals for continuous outcomes (adjusted relative percentage change 8.35%, 95% CI 7.46% to 9.24%; median 2.00%; interquartile range ‐1.00% to 21.00%))

The certainty of evidence for this comparison is moderate.

Educational meetings alone compared with other interventions

• May improve compliance with desired practice when compared with other interventions (6 studies, 1402 health professionals for dichotomous outcomes (adjusted risk difference 9.99%, 95% CI 9.47% to 10.52%; median 16.5%; interquartile range 0.80% to 16.50%); 2 studies, 72 health professionals for continuous outcomes (adjusted relative percentage change 12.00%, 95% CI 9.16% to 14.84%; median 12.00%; interquartile range 0.00% to 24.00%))

No studies met the inclusion criteria for patient outcome measurements. The certainty of evidence for this comparison is low.

Interactive educational meetings compared with didactic (lecture‐based) educational meetings

• We are uncertain of effects on compliance with desired practice (3 studies, 370 health professionals for dichotomous outcomes; 1 study, 192 health professionals for continuous outcomes) or on patient outcomes (1 study, 54 health professionals for continuous outcomes), as the certainty of evidence is very low

Any other comparison of different formats and durations of educational meetings

• We are uncertain of effects on compliance with desired practice (1 study, 19 health professionals for dichotomous outcomes; 1 study, 20 health professionals for continuous outcomes) or on patient outcomes (1 study, 113 health professionals for continuous outcomes), as the certainty of evidence is very low.

Factors that might explain heterogeneity of effects

Meta‐regression suggests that larger estimates of effect are associated with studies judged to be at high risk of bias, with studies that had unit of analysis errors, and with studies in which the unit of analysis was the provider rather than the patient.

Improved compliance with desired practice may be associated with: shorter meetings; poor baseline compliance; better attendance; shorter follow‐up; professionals provided with additional take‐home material; explicit building of educational meetings on theory; targeting of low‐ versus high‐complexity behaviours; targeting of outcomes with high versus low importance; goal of increasing rather than decreasing behaviour; teaching by opinion leaders; and use of didactic versus interactive teaching methods.

Pre‐specified exploratory analyses of behaviour change techniques suggest that improved compliance with desired practice may be associated with use of a greater number of behaviour change techniques; goal‐setting; provision of feedback; provision for social comparison; and provision for social support. Compliance may be decreased by the use of follow‐up prompts, skills training, and barrier identification techniques.

Authors' conclusions

Compared with no intervention, educational meetings as the main component of an intervention probably slightly improve professional practice and, to a lesser extent, patient outcomes. Educational meetings may improve compliance with desired practice to a greater extent than other kinds of behaviour change interventions, such as text messages, fees, or office systems. Our findings suggest that multi‐strategy approaches might positively influence the effects of educational meetings.

Additional trials of educational meetings compared with no intervention are unlikely to change the review findings; therefore we will not further update this review comparison in the future. However, we note that randomised trials comparing different types of education are needed.

Plain language summary

Continuing education meetings and workshops: effects on healthcare professionals’ practice and on patients’ health

What is the aim of this review?

The aim of this Cochrane Review was to assess the effects of educational meetings on healthcare professionals’ practice and on patients’ health. Review authors searched for all relevant studies to answer this question and included 215 studies in the review.

Key messages

Educational meetings alone or as the main part of a larger strategy are probably better than no strategy for improving healthcare professionals’ practice and patients' health. They also may be better than other types of behaviour change strategies for improving healthcare professionals’ practice. But we do not know if some types of educational meetings are better than others.

What was studied in the review?

Educational meetings include courses, seminars, and workshops in various formats. Doctors and other healthcare professionals often use educational meetings as part of their continuing medical education. Medical societies and employers in the healthcare system also use educational meetings to present new knowledge or new types of care and to encourage best practice. These types of meetings can vary a lot. For instance, some may be very interactive, and other may be lecture‐based. Types of people leading the meetings and numbers of people who attend also vary.

But do these types of meetings lead to change? The review authors assessed whether healthcare professionals who went to educational meetings were more likely to follow practices recommended to them. In addition, review authors assessed whether these meetings led to any improvements in patients’ health.

This review is an update of an earlier Cochrane Review.

What are the main results of the review?

Review authors included 215 relevant studies involving more than 28,000 healthcare professionals.

Most of the studies were from North America or Europe, although many other countries were also represented. Most studies took place in primary care or community‐based care settings such as nursing homes, but many took place in hospitals and other secondary care settings. Most of the healthcare professionals in these studies were doctors, but the studies looked at other groups, including nurses, pharmacists, physiotherapists, and dentists. This review shows the following.

• Educational meetings alone or as the main part of a larger package, compared with no meetings

‐ Healthcare professionals are probably more likely to follow recommended practices (moderate‐certainty evidence)

‐ This probably slightly improves patient health (moderate‐certainty evidence)

• Educational meetings alone compared with other strategies to change healthcare professionals’ behaviour

‐ Healthcare professionals may be more likely to follow recommended practices (low‐certainty evidence)

‐ We do not know about effects on patient health because we found no relevant studies

• Interactive educational meetings compared with lecture‐based educational meetings

‐ We do not know about effects on healthcare professionals’ practice or on patients’ health because the certainty of evidence is very low

• Any other comparison of different types of educational meetings

‐ We do not know about effects on healthcare professionals’ practice or on patients’ health because the certainty of evidence is very low

How up‐to‐date is this review?

The review authors searched for studies in November 2016.

Summary of findings

Background

Educational meetings are among the most common continuing medical education activities offered to healthcare professionals (Brown 2002; Lloyd 1979). Educational meetings can include courses and workshops in various formats. The nature of educational meetings is highly variable in terms of content, number of participants, degree and type of interaction, length, frequency, and targeted practices. Other common continuing medical education activities are audit and feedback and educational outreach (Ivers 2012; O'Brien 2007), both of which are frequently combined with educational meetings. Quality improvement activities, which are closely related to continuing education (Boonyasai 2007), often are presented in small interactive meetings, to facilitate learning and practice improvements. Reeves 2013 investigated the effects of interprofessional education in particular.

An underlying assumption is that continuing medical education can improve healthcare practice and, thereby, health outcomes for patients. Two overviews of reviews on continuing medical education in general concluded that continuing medical education can be effective (Bloom 2005; Umble 1996), but that the effects are highly variable. The first overview was based on 16 reviews conducted between 1984 and 1994, and the other on 26 reviews prepared from 1984 to 2004. From the late 1980s, questions of how and why some programmes worked better than others were raised and investigators looked for potential explanatory factors. The focus of researchers shifted from measuring knowledge, attitudes, or skills to measuring physicians' performance or patients' health. Commonly reported findings from explanatory analyses were that interventions using an interactive educational format had greater effect than those using a didactic format, and that multi‐faceted interventions had greater effect than single interventions (Mansouri 2007; Marinopoulos 2007).

Earlier versions of this review assessed the impact of educational meetings and examined factors that could explain variations in effectiveness (Davis 1999; O'Brien 2001). These review authors concluded that interactive workshops can result in moderately large changes in professional practice but didactic sessions alone are unlikely to change professional practice. Another review of a wide range of guideline implementation strategies concluded that educational meetings, with or without educational material, resulted in small to modest improvements when compared with no intervention (Grimshaw 2004).

In our previous version of this review (Forsetlund 2009), we examined the effects of continuing medical education meetings on professional practice and patient outcomes. We also investigated factors that might influence the effectiveness of educational meetings. Based on 81 included studies, we concluded that educational meetings alone or combined with other interventions can improve professional practice and patients' achievement of treatment goals. The effect on professional practice tended to be small but varied between studies, and the effect on patient outcomes was generally less, as would be expected. It was not possible to explain observed differences in effect with confidence, but it appeared that higher attendance at meetings was associated with greater effects, that mixed interactive and didactic education was more effective than either alone, and that effects were less for more complex behaviours and less serious outcomes.

Given that the number of publications examining educational meetings has been increasing and that uncertainty as to what causes variation in effects of educational meetings has been ongoing, we decided to update this review. For this update, we considered additional factors that might explain the variation in results of educational meetings. We based the selection of explanatory factors on both previous literature on continuing medical education (e.g. Davis 1992, Davis 1999, Mansouri 2007, Marinopoulos 2007) and theories of professional behaviour change (e.g. Locke 2002, Sniehotta 2009, Wensing 2005).

Description of the condition

Slow diffusion and implementation of research results (Balas 2000), large variations in professional practice (Mays 2011; Wennberg 2011), and major deficiencies in patient safety represent constant challenges in the healthcare delivery system (Institute of Medicine 2001). For example, studies have consistently found gaps between recommended practice and actual practice, although the size of the gap is variable (Campbell 2007; Grimshaw 2001; Grimshaw 2004; Grol 2003; McGlynn 2003; Rabe 2000). In the previous version of this review, we found that the median baseline adherence to recommended practices was only 40% (interquartile range 18% to 57%). To improve healthcare services, one of the important tasks at hand is to identify effective strategies for improving and maintaining healthcare providers' professional performance.

Description of the intervention

Educational meetings are among several continuing medical education activities that "serve to maintain, develop, or increase the knowledge, skills, and professional performance and relationships that a physician uses to provide services for patients, the public or the profession" (American Medical Association 2006). Educational meetings include courses, seminars, and workshops in various formats. The nature of educational meetings varies in terms of aims, content, number of participants, degree and type of interaction, length, frequency, and types of practices targeted. Other common continuing medical education activities are audit and feedback and educational outreach (Ivers 2012 O'Brien 2007), both of which are frequently combined with educational meetings. Broad overviews of continuing medical education and guideline implementation strategies have found that they, and educational meetings in particular, can be effective. However, effectiveness varies between settings (Bero 1998; Bloom 2005; Grimshaw 2001; Grimshaw 2004; Umble 1996).

Previous systematic reviews have investigated factors that might explain observed variation in the effects of educational meetings. Findings from these analyses include the following.

Interventions using an interactive educational format may have greater effects on health provider practice than those using a didactic format (Davis 1992; Davis 1999; Mansouri 2007; Marinopoulos 2007; O'Brien 2001).
Interventions using single live media or several types of media may have greater effects than those using single print media (Marinopoulos 2007).
Interventions that are appropriately implemented ‐ measured as proportion of participants attending the educational meeting(s) ‐ may have greater effect than those inadequately implemented (Forsetlund 2009).
Interventions targeting outcomes that are likely to be perceived as serious may be more effective (Forsetlund 2009).
Interventions targeting more complex behaviours may be less effective (Forsetlund 2009; O'Brien 2001).
Interventions targeted at participants from the same discipline (versus multiple disciplines) may have larger effects (Mansouri 2007).
Interventions with fewer rather than more participants may have larger effects (Mansouri 2007).
Interventions with a beforehand assessment of needs or barriers may be associated with larger effects (Davis 1992; Davis 1995).
More, rather than fewer, sessions may increase the effectiveness of an intervention (Mansouri 2007); this finding was not replicated in another review (Marinopoulos 2007).
Interventions of longer duration (measured as contact hours) may have larger effects than those of shorter duration (Mansouri 2007).
Shorter, rather than longer, time for follow‐up was associated with greater effects (Mansouri 2007); this finding was not replicated in another review (Marinopoulos 2007).
Although some reviews have found that multi‐faceted interventions may be more effective than educational interventions alone (Davis 1995; Mansouri 2007; Marinopoulos 2007), the findings of other reviews have not supported this conclusion (Forsetlund 2009; Grimshaw 2004; Ivers 2012; O'Brien 2007).
Educational interventions facilitating rehearsal of practice behaviours may have larger effects than those not providing skills training (Davis 1992).

Other potential explanatory factors that we explored in the previous version of this review could not explain the variation in effectiveness of educational meetings. These included baseline compliance, risk of bias, and meeting intensity (a judgement that we made based on combined measure of several characteristics) (Forsetlund 2009).

Several investigators have reviewed behavioural change theories for changing professional practice and by this approach have identified sets of constructs or taxonomies used to detect barriers and facilitators of change and behaviour change techniques (Abraham 2008; Grol 2007; Michie 2005; Michie 2008; Wensing 2005). In addition, Michie 2010 developed a method for assessing the extent to which behavioural interventions are theory‐based, and Gearing 2011 created a tool for measuring degree of intervention fidelity. In updating this review, we have considered factors explored in previous systematic reviews, factors identified by reviews of behavioural change theories, and factors related to tools for assessing the theoretical basis and fidelity of an intervention. For each factor, we have considered the basis for the hypothesis, whether it was likely that included studies would provide data, whether we could reliably code included studies, and the hypothesised direction of effect (i.e. whether the factor would increase or decrease the effectiveness of educational meetings). Potential explanatory factors that we elected to include in the updated version of this review based on these considerations are summarised in Appendix 1. Potential explanatory factors that we considered and did not include along with reasons for excluding them are summarised in Appendix 2.

This review addressed the following questions.

Are educational meetings and workshops, alone or supported by other interventions, effective in improving professional practice or healthcare outcomes?

Comparison 1. Educational meetings as the main component of the intervention compared with no intervention
The aims of this analysis were to investigate the effects of educational meetings on health professionals' practice and to explore heterogeneity. From results of the previous review, we expected considerable variation in current results. We chose explanatory factors for consideration on the basis of cross‐theory literature reviews for identifying important factors in behaviour change, systematic reviews of interventions for behaviour change, a taxonomy for coding intervention contents (Abraham 2008), a coding scheme for use of theory and degree of implementation (Borrelli 2005; Michie 2010). and which study data that we believed would be available (see section on Assessment of heterogeneity, Appendix 1, and Appendix 3 for more information). Another consideration was that the number of explanatory factors in the multi‐variable meta‐regression model must be proportionate to the number of total comparisons

How does the effectiveness of educational meetings compare with that of other interventions?

Comparison 2. Educational meetings alone compared with other interventions

Can educational meetings be made more effective by modifying how they are done?

Comparison 3. Interactive educational meetings compared with didactic (lecture‐based) educational meetings
Comparison 4. Any other comparison of different types of educational meetings (e.g. different meeting durations, other formats)

We used methods that have been developed by the Cochrane Effective Practice and Organisation of Care (EPOC) Group (Grimshaw 2003). Because printed materials are usually an integral part of educational meetings, we chose to consider printed educational materials as a component of educational meetings, and not as an additional independent intervention. Few studies have tested educational meetings with no printed educational materials (Grimshaw 2004).

How the intervention might work

Current standards and efforts to improve reporting of educational interventions have been criticised (Horsley 2018). In our previous review, few study authors described explicitly how they imagined the educational intervention might work, or which theories and frameworks guided them in preparation of the intervention. Generally, study authors described the intervention in terms of what they sought to teach and practicalities pertaining to this but were silent on describing any underlying theory or logical explanation of how the intervention might work. This is highly problematic notwithstanding the challenges imposed on consumers of these trials who will struggle with replication or implementation.

In general, the implicit underlying assumption is that transferring knowledge to healthcare professionals through educational activities such as workshops improves their practice and skills and thereby results in better health outcomes for patients. We, as authors, recognise how insufficient this is. It is unclear how a change in knowledge would lead to a change in behaviour. An often used model for illustrating the presumed causal chain for behaviour change among humans in general is the theory of planned behaviour, which is based on social cognitive theory (Armitage 2001; Eccles 2006; Hardeman 2002). According to this model, the success of an educational meeting in changing behaviour would depend not only on its ability to communicate and convey knowledge and skills to the individual but also on its ability to change several mediating factors in the causal chain for a behaviour to be performed. The educational meeting would have to create and increase positive attitudes (outcome expectancy) toward the targeted behaviour, a belief that the behaviour is consistent with or desired by important others (social or subjective norms), and a feeling of perceived behavioural control (related to self‐efficacy) (Walker 2003). These three mediators are supposed to influence the strength of the intention to perform the desired behaviour, which in turn will influence whether the actual behaviour is executed.

We assessed the extent to which interventions were theory‐based by using one of the indicators from the method of Michie 2010, and we coded the active ingredients (behaviour change techniques) in each intervention using Abraham and Michie’s taxonomy (Table 5) (Abraham 2008). We estimated whether inclusion of a larger number of these behaviour change techniques is associated with larger effects (Table 6). We used the number of participants attending the educational meeting(s) as an indicator for intervention implementation (Gearing 2011).

1. Behaviour change techniques used in the included studies.

Explanatory factors based on behaviour change techniques used in included studies (Abraham 2008&Michie 2008)	Number of studies using these techniques (%)
Barrier identification (technique 5)	29 (13)
Use of written goal‐setting, action‐planning, or behavioural contracts (technique 4, 10, or 16)	16 (7)
Provision of feedback on performance (technique 13)	60 (28)
Provision of opportunities for social comparison (e.g. opinion leaders; technique 19)	8 (3)
Skills training/Prompting of practice (technique 8 or 17)	79 (37)
Use of follow‐up prompts (reminders in any form such as telephone contacts or pop‐up screens; technique 18)	61 (28)
Social support or social change (item 20)	11 (5)

Targeted behaviour	No. of studies
Preventive care ‐ Identifying and managing problems in marital relationships; smoking cessation counselling; breastfeeding promotion activities; nutrition, exercise, and health behaviours counselling; infection control; HIV prevention; prevention of falls in nursing homes; screening (cancer, lipids, identification of maltreatment of children or partner); follow up of patients with coronary artery disease.	32
Test ordering behaviour	6
Screening behaviours for colorectal, skin, breast, prostatic, and cervical cancer; dementia; arthritis; mental health or psychosocial problems	10
Prescribing ‐ Reducing antibiotic, steroid, or NSAID prescribing; optimisation of prescribing of beta‐blockers; appropriate prescribing of drugs for acute diarrhoea; asthma; upper respiratory tract infection; newly detected heart failure; elderly people in general	36
General management of a wide array of conditions ‐ Management of low back pain, urinary tract infection, stroke rehabilitation, sore throat, sexually transmitted disease, obstetrical practices, preterm delivery, tuberculosis, obesity, asthma, osteoarthritis, neck pain, diabetes, angina, acute myocardial infarction, congestive heart failure, epilepsy, infertility, neonatal care, hypercholesterolaemia, mental disorder or attention deficit hyperactivity disorder	87
Communication skills	39
Other types of targeted behaviour ‐ Clinical interventions by pharmacist, diagnosing depression in the elderly, use of spirometry, root filling of teeth, early referral to mental health services	5

Comparison 1. Educational meetings as a single intervention or as the main component of a multi‐faceted intervention compared with no intervention
Patient or population: health personnel Settings: health care Intervention: interventions with educational meeting(s) as the main component Comparison: no intervention
Outcomes	*Illustrative comparative compliances and means (IQR) or [95% CI]**		Relative effect (IQR) or [95% CI]	No. of participants (studies)	Certainty of evidence (GRADE)	Results in words
	Assumed	Corresponding
	No meetings	Meetings
Health professional outcomes
Compliance with desired practice (Yes/No ‐ dichotomous outcomes			Adjusted risk difference			Inclusion of educational meetings as the main component of an intervention probably slightly improves compliance with desired practice, compared with no intervention
Median			4.00% (0.29% to 13.00%)	7868 (65)	⊕⊕⊕⊝ MODERATE^a,b
Mean	500 of 1000 healthcare professionals	568 [566 to 570] of 1000 healthcare professionals	6.79% [6.62% to 6.97%]	7868 (65)	⊕⊕⊕⊝ MODERATE^a,b
Compliance with desired practice (measured on a continuous scale)			Adjusted relative % change
Median			20.00% (6.00% to 65.00%)	2360 (27)	⊕⊕⊕⊝ MODERATE^a,b
Mean	5.0 (on a scale of 0 to 10)^c	6.0 [5.3 to 8.3] (on a scale of 0 to 10)	44.36% [41.98% to 46.75%]	2360 (27)	⊕⊕⊕⊝ MODERATE^a,b
Patient outcomes
Dichotomous			Adjusted risk difference			Inclusion of educational meetings as the main component of an intervention probably slightly improves patient outcomes, compared with no intervention
Median			0.10% (0.00% to 4.00%)	2530 (15)	⊕⊕⊕⊝ MODERATE^a,b
Mean	300 of 1000 patients	333 [331 to 335] of 1000 patients	3.30% [3.10% to 3.51%]	2530 (15)	⊕⊕⊕⊝ MODERATE^a,b
Continuous			Adjusted relative % change
Median			2.00% (‐1.00% to 21.00%)	2294 (28)	⊕⊕⊕⊝ MODERATE^a,b
Mean	3.0 (on a scale of 0 to 10)^c	3.3 [3.2 to 3.3] (on a scale of 0 to 10)	8.35% [7.46% to 9.24%]	2294 (28)	⊕⊕⊕⊝ MODERATE^a,b
The method used to compute assumed* and corresponding compliances and means is described in Data collection and analysis (see Summary of findings and assessment of the certainty of evidence). CI: Confidence interval; IQR: interquartile range (presented as an interval).
GRADE Working Group grades of evidence. High certainty: this research provides a very good indication of the likely effect. The likelihood that the effect will be substantially different is low. Moderate certainty: this research provides a good indication of the likely effect. The likelihood that the effect will be substantially different is moderate. Low certainty: this research provides some indication of the likely effect. However, the likelihood that it will be substantially different is high. Very low certainty: this research does not provide a reliable indication of the likely effect. The likelihood that the effect will be substantially different is very high. ** Substantially different: a large enough difference that it might affect a decision

Comparison 2. Educational meetings alone compared with other interventions
Patient or population: health personnel Settings: health care Intervention: educational meetings Comparison: other interventions
Outcomes	*Illustrative comparative compliances and means (IQR) or [95% CI]**		Relative effect (IQR) or [95% CI]	No. of participants (studies)	Certainty of evidence (GRADE)	Results in words
	Assumed	Corresponding
	Other interventions	Meetings
Health professional outcomes
Compliance with desired practice (Yes/No ‐ dichotomous outcomes)			Adjusted risk difference			Educational meetings alone may improve compliance with desired practice, compared with other interventions
Median			16.50% (0.80% to 16.50%)	1402 (6)^a	⊕⊕⊝⊝ Low^b,c
Mean	500 of 1000 healthcare professionals	600 [594 to 605] of 1000 healthcare professionals	9.99% [9.47% to 10.52%]
Compliance with desired practice (measured on a continuous scale)			Adjusted relative % change
Median			12.00% (0.00% to 24.00%)	72 (1)^d	⊕⊕⊝⊝ Low^b,c
Mean	5.0 (on a scale of 0 to 10)^e	5.6 [5.5 to 5.7] (on a scale of 0 to 10)	12.00% [9.16% to 14.84%]
Patient outcomes
No results met the inclusion criteria for this comparison.
The method used to compute assumed* and corresponding compliances and means is described in Data collection and analysis (see Summary of findings and assessment of the certainty of the evidence). CI: Confidence interval; IQR: Interquartile range (presented as an interval).
GRADE Working Group grades of evidence. High certainty: this research provides a very good indication of the likely effect. The likelihood that the effect will be substantially different is low. Moderate certainty: this research provides a good indication of the likely effect. The likelihood that the effect will be substantially different is moderate. Low certainty: this research provides some indication of the likely effect. However, the likelihood that it will be substantially different is high. Very low certainty: this research does not provide a reliable indication of the likely effect. The likelihood that the effect will be substantially different is very high. ** Substantially different: a large enough difference that it might affect a decision

Comparison 3. Interactive educational meetings compared with didactic (lecture‐based) educational meetings
Patient or population: health personnel Settings: health care Intervention: interactive educational meetings Comparison: didactic educational meetings
Outcomes	*Illustrative comparative compliances and means (IQR) or [95% CI]**		Relative effect (IQR) or [95% CI]	No. of participants (studies)	Certainty of evidence (GRADE)	Results in words
	Assumed	Corresponding
	Didactic meetings	Interactive meetings
Health professional outcomes
Compliance with desired practice (Yes/No ‐ dichotomous outcomes)			Adjusted risk difference			We are uncertain of the effects on compliance with desired practice of interactive educational meetings compared with didactic educational meetings as the certainty of evidence is very low
Median			4.50% (1.40% to 4.50%)	370 (3)^a	⊝⊝⊝ Very low^b,c
Mean	500 of 1000 healthcare professionals	542 [539 to 545] of 1000 healthcare professionals	4.18% [3.87% to 4.49%]	370 (3)^a	⊝⊝⊝ Very low^b,c
Compliance with desired practice (measured on a continuous scale)			Adjusted relative % change
Median and mean	5.0 (on a scale of 0 to 10)^d	8.0 (on a scale of 0 to 10)	60.00%	192 (1)^e	⊝⊝⊝ Very low^b,c
Patient outcomes
Dichotomous	No results met the inclusion criteria
Continuous			Adjusted relative % change			We are uncertain of the effects on patient outcomes of interactive educational meetings compared with didactic educational meetings as the certainty of the evidence is very low
Improvement in patients' low‐density lipoprotein cholesterol (median and mean^f)	3.0 (on a scale of 0 to 10)^d	3.4 (on a scale of 0 to 10)	12.00%	54 (1)^g	⊝⊝⊝ Very low^b,c
The method used to compute assumed* and corresponding compliances and means is described in Data collection and analysis (see Summary of findings and assessment of the certainty of the evidence). CI: confidence interval; IQR: interquartile range (presented as an interval).
GRADE Working Group grades of evidence. High certainty: this research provides a very good indication of the likely effect. The likelihood that the effect will be substantially different is low. Moderate certainty: this research provides a good indication of the likely effect. The likelihood that the effect will be substantially different is moderate. Low certainty: this research provides some indication of the likely effect. However, the likelihood that it will be substantially different is high. Very low certainty: this research does not provide a reliable indication of the likely effect. The likelihood that the effect will be substantially different is very high. ** Substantially different: a large enough difference that it might affect a decision

Comparison 4. Any other comparison of different types of educational meetings
Patient or population: health personnel Settings: health care Intervention: educational meeting Comparison: another kind of educational meeting
Outcomes	*Illustrative comparative compliances and means (IQR) or [95% CI]**		Relative effect (IQR) or [95% CI]	No. of participants (studies)	Certainty of evidence (GRADE)	Results in words
	Assumed	Corresponding
	One type of educational meeting	Another type of educational meeting
Health professional outcomes
Compliance with desired practice (Yes/No ‐ dichotomous outcomes)	Less intensive education	More intensive education	Adjusted risk difference			We are uncertain of the effects on compliance with desired practice of educational meetings compared with other kinds of educational meetings as the certainty of the evidence is very low
Median and mean	500 of 1000 healthcare professionals receiving less intensive education	619 of 1000 healthcare professionals receiving more intensive education	11.90%	19 (1)^a	⊝⊝⊝⊝ Very low^b,c
Compliance with desired practice (measured on a continuous scale)	Educational meetings focused on critical appraisal	Educational meetings using problem‐based strategies	Adjusted relative % change
Median and mean	5.0 (on a scale of 0 to 10)^d	5.0(on a scale of 0 to 10)	‐1.00%	20 (1)^e	⊝⊝⊝⊝ Very low^b,f
Patient outcomes
Dichotomous	No results met the inclusion criteria.
Continuous	Shorter educational meetings	Longer educational meetings	Adjusted relative % change			We are uncertain of the effects on patient outcomes of educational meetings compared with other types of educational meetings as the certainty of the evidence is very low
Median and mean (patients' self‐reported distress score )	3.0 (on a scale of 0 to 10)^d	4.0 (on a scale of 0 to 10)	34.00%	113 (1)^g	⊝⊝⊝⊝ Very low^b,f
The method used to compute assumed* and corresponding compliances and means is described in Data collection and analysis (see Summary of findings and assessment of the certainty of the evidence). CI: confidence interval; IQR: interquartile range (presented as an interval).
GRADE Working Group grades of evidence. High certainty: this research provides a very good indication of the likely effect. The likelihood that the effect will be substantially different is low. Moderate certainty: this research provides a good indication of the likely effect. The likelihood that the effect will be substantially different is moderate. Low certainty: this research provides some indication of the likely effect. However, the likelihood that it will be substantially different is high. Very low certainty: this research does not provide a reliable indication of the likely effect. The likelihood that the effect will be substantially different is very high. ** Substantially different: a large enough difference that it might affect a decision

Date	Event	Description
27 August 2020	New citation required and conclusions have changed	142 new studies are included in this update. The total number of included studies in the review is now 215. New authors contributed to this update
2 July 2020	New search has been performed	This is the second update of the Cochrane Review first published in 2001. A new search was conducted and other content updated
11 October 2012	Amended	Reference for Orstein 2004 corrected
12 February 2009	New citation required and conclusions have changed	This is an update of previously published review. Forty‐nine new studies have been added to the 32 studies from the previous review, making a total of 81 included studies. The search was re‐run in December 2007: Seventy‐seven references are listed under 'Studies awaiting classification'.
12 February 2009	New search has been performed	All searches updated.
19 June 2008	Amended	Converted to new review format.

Explanatory factors	Hypothesis (larger effects associated with)	Basis for hypothesis	Operationalisation
Number of behaviour change techniques	Higher number	Several previous reviews have explored multi‐faceted vs single interventions and found little or no difference in effect between them (Squires 2014). Multi‐faceted interventions are related to but are not the same as using multiple behaviour change techniques In general, it is assumed that playing on several sense impressions and targeting several of the constructs and stages of the causal chain for behaviour change should be more successful for processing and retaining new information and for enabling execution of new behaviour than when only 1 technique is used	We coded all behaviour change techniques used in the intervention, according to Abraham 2008. This is a theory‐linked taxonomy for classification of all components of an intervention. The techniques are targeted at theoretical constructs of the causal chain described by behavioural change theories for explaining behaviour change. 'Number of behaviour change techniques' is the sum of all techniques that were used in 1 study
Provision of take‐home materials to support implementing changes in practice	Provision better than no provision	Use of different media to transfer the message would target more senses and would give opportunity for repetition of main messages, which could help coding and processing of new information into long‐term memory	Any take‐home materials vs none. We did not code for printed materials used during teaching but only when additional materials to take home such as laminated cards, guidelines, protocols, etc., were used
Duration	More hours	Length of exposure in terms of number of hours and frequency is seen as important because it gives greater opportunity to persuade, to repeat for improved memorising and maturing of the message, and to use several techniques targeted at several senses, although empirical data supporting this are sparse (O'Neil 2009)	Measured as number of total hours of duration (contact time). If duration was reported as number of days, we estimated 1 day as 8 hours and a half‐day as 4 hours
Attendance	More attendance	Logic (if people do not attend, they are unlikely to be affected). This factor is chosen as an indicator of the extent of implementation: “implementation is incomplete if a program is unable to attract and retain a high percentage of its target population in the intervention” (Durlak 1998)	If reported by study authors, we recorded the percentage of study participants that actually attended the educational session(s); if not reported, we estimated attendance on the basis of information in the text. If this was not possible, attendance was recorded as unknown
Use of theory	Intervention explicitly based on theory	Theories provide a framework for identifying factors that determine behaviour and for selecting appropriate interventions to address those factors (Michie 2010) A review of studies using the Internet to promote health behaviour found that increased use of theory was associated with larger effect sizes (Webb 2010)	Michie 2010 published a tool for deciding to which extent theory was used to guide intervention development. We used only item 5 from this tool: "the intervention is explicitly based on a theory or predictor or combination of theories or predictors"
Interactive vs passive (didactic)	Interactive	In the cognitive perspective, teaching should emphasise to strengthen motivation and attention, stimulate curiosity, encourage activity and interaction, make sure the new information relates to past experiences, play on multiple senses, and create opportunities for reflection. The teacher should identify when learning is occurring – possible only when participants are allowed to be active – and should respond with a reinforcing behavior (social cognitive theories; Knowles 2005)	We classified educational meetings as interactive or didactic. Didactic sessions were defined as those that were predominantly lectures or presentations but may have included question and answer periods. Interactive workshops and seminars were defined as sessions that involved some type of interaction amongst participants. Studies that reported using such techniques as role‐play, discussion groups, problem‐based learning, or androgogical methods. or that were case‐ or vignette‐based, we categorised as interactive. CHEST guidelines classified educational techniques as case‐based learning, demonstration, discussion group, role‐play, etc., but did not find any detectable difference in effect between them (O'Neil 2009). This may support the idea that the most important divide is between interactivity and passivity. Interventions that mixed the 2 formats were classified as interactive
Complexity	Less complex behaviours	Complexity of the desired new behaviour is supposed to be one of the main factors influencing whether the innovation is adopted in a social system or not (social cognitive theories ‐ Locke 2002; Rogers 1995 ‐ diffusion of innovations + logic)	The complexity of the targeted behaviour (high or low) was categorised in a subjective manner independently by 2 of us. The categories depended upon the number of behaviours required, the extent to which complex judgements or new skills were necessary, whether other factors such as organisational change were required for the behaviour to be improved, whether there was need for change only by the individual/professional (one person) or communication change or system change (need for new equipment, payment, more time), and whether there was reason to expect strong patient expectations
Importance	Important (clinicians’ perceived importance to patients; good use of their time)	Clinicians more likely to put more effort into change for things that they perceive as important or advantageous for the patient (social cognitive theories ‐ Rogers 1995 + logic)	The importance of the outcome (high or low) was categorised in a subjective manner independently by 2 of us. Acute problems with high risk of serious consequences (acute or secondary prevention) were considered as high. Primary prevention with a long‐term perspective was considered as low. The seriousness of testing and prescribing could be assessed as low or high, depending on the seriousness of the suspected or existing diagnosis
Baseline compliance	Lower baseline compliance	Logic (greater potential for change + may be a ceiling effect with high baseline compliance)	Baseline compliance with targeted behaviours was treated as a continuous variable ranging from zero to 100%, based on the pre‐intervention level of compliance as a mean for both or all groups before the intervention
Risk of bias	High risk of bias	Most biases are in favour of the intervention	Three categories from RoB (low, unclear, high)
Time to follow‐up	Could be in either direction	Social cognitive theory: time is important for allowing information processing, repetition, and maturing. On the other hand, without repetition or reminders, passing of time could be destructive to performance of targeted behaviour	Measured as time passed from end of intervention to measurement
Unit of analysis errors	More precise estimates (i.e. overly narrow confidence intervals) associated with unit of analysis error. This occurs when studies are analysed at a different level than the level of allocation (e.g. individual patient level rather than cluster level), without proper adjustment of the analysis	Not accounting for intra‐cluster correlation (relatedness of responses within a cluster) when using other units as the unit of analysis rather than the allocation unit is likely to lead to artificially narrow confidence intervals (Cochrane Handbook for Systematic Reviews of Interventions, Chapter 23. Including variants on randomized trials: https://training.cochrane.org/handbook/current/chapter‐23 https://www.bmj.com/content/345/bmj.e5661)	We noted any unit of analysis error for each study when data were collected
Unit of analysis	Studies for which the effects of an intervention are measured at the same level as that to which the intervention was directed (i.e. the provider level). This is in contrast to studies that measure effects at a different level (i.e. the patient level)	It is more likely that continuing education meetings will impact the behaviours of providers who participate in these meetings, and less likely that they will impact patient behaviours and outcomes. This is so because patients are not directly targeted by these interventions – rather, effects on patients are mediated through the provider	We noted unit of analysis for each study when data were collected

Factor	Hypothesis (larger effects associated with)	Basis for hypothesis	Operationalisation	Comments
Additional modes of information delivery	Additional modes	Could target more senses: printed material, videos (social cognitive theories)	Mode = not sure EM ± printed materials	Drop – overlaps with other factors (e.g. techniques of change such as item 9 in Abraham’s taxonomy) and would be difficult to code other than ± printed/other materials
Seriousness	More serious	Social cognitive theory + logic	Change from 3 categories to 2 Seriousness = high risk (treatment of an acute problem or secondary prevention) of severe consequences (availability of an effective intervention that clinicians can and should deliver)	Change to importance (see Additional table 1)
Hospital setting vs primary care	Hospital setting	More opportunity for modelling and organisational support/reinforcement/social network (social cognitive theories; Rogers 1995). Also greater complexity of behaviour, which may pull in opposite direction	Easy to code but not easy to interpret. Do not include in analysis	Code used descriptively only
Country setting	May be different effects in low‐ vs middle‐ vs high‐income countries	No obvious basis for saying what direction	WB categories	Categories used descriptively and applicability of findings to LIC discussed
Clinical area	None	None		Descriptive only
Targeted behaviour	None	Addressed by complexity and importance		Descriptive only

Explanatory factor, based on behaviour change techniques used in interventions tested in the included studies and coded according to the taxonomy in Abraham 2008	Hypothesis (larger effects associated with)	Basis for hypothesis	Operationalisation
Barrier identification (technique 5)	Reduced barriers	Linked to social cognitive theories, which hypothesise that barriers may hinder behaviour even if intentions are strong (Abraham 2008; Wensing 2005)	This item is defined as "Identify barriers to performing the behavior and plan ways of overcoming them" (item 5). The original meaning may have been to prompt participants to identify barriers, but we will use this item to code for identification of barriers by investigators prior to the study to tailor the intervention
Use of written goal‐setting or action‐planning (behavioural contracts) (technique 4, 10, 16)	Formally signed contracts about behavioural goals	The importance of goal‐setting is central to several social cognitive theories (e.g. goal‐setting theory (Locke 2002), diffusion of innovations (Rogers 1995)). Formally signed contracts about behavioural goals are believed to strengthen intentions to fulfil them (Michie 2008; Sniehotta 2009)	The taxonomy of Abraham 2008 is somewhat unclear on whether it distinguishes between experimenter‐ and participant‐initiated goal‐setting contracts or action plans (Sniehotta 2009). Compared with Michie 2008, it is also unclear whether action‐planning is considered a theoretical construct domain that may be influenced by goal‐contracting, rather than a behaviour change technique for influencing intentions. We did not distinguish between goal‐setting and action‐planning concepts. We hypothesised that such a level of detail would not be reached by the included studies. We decided beforehand to code studies that reported prompting of 'intention formation' (technique 4), 'specific goal‐setting' (technique 10), and 'behavioural contracts' (technique 16) as using intention and motivation strengthening techniques
Provision of feedback on performance (technique 13)	Feedback	Abraham 2008 links the construct to control theory. It may also map on to social cognitive theories: feedback is an important response from the environment that the individual interacts with and it may act as a reinforcer for the individual’s self‐regulation)	We defined this item as "providing data about recorded behavior or evaluating performance in relation to a set standard or others' performance, i.e. the persons received feedback on their behavior". In Abraham 2008, the item is linked to control theory, but it might as well have been linked to goal‐setting theory: "For goals to be effective, people need summary feedback that reveals progress in relation to their goals" (Locke 2002)
Provide opportunities for social comparison (e.g. opinion leaders; technique 19)	Social comparison	Abraham 2008 links this construct to social cognitive theory: Gives the opportunity for learning by modelling others and being influenced by social stimuli in the environment	In the taxonomy, this item was defined as "facilitate observation of nonexpert others' performance, for example, in a group class or using video or case study" (item 19). We will consider the use of opinion leaders in other roles than as the teacher as a technique for facilitate social comparison
Use of follow‐up prompts (reminders in any form such as telephone contacts or pop‐up screens; technique 18)	Reminders	Abraham 2008 does not specify the theoretical background, but this construct maps on to social cognitive theories, which underscores the importance of repetition of a message until it is incorporated into long‐term memory.	This is item 18 in the taxonomy: "contacting the person again after the main part of the intervention is complete". For example, distribution of newsletters, establishment of help desks and services, use of telephone contacts, or use of any kind of reminders such as computerised pop‐up screens we coded as use of follow‐up prompts
Prompting of practice (by this we mean skills‐training; technique 17)	Skills training/Prompting of practice	Abraham 2008 links this construct to Skinner’s theory of operant conditioning.	This is item 17 in the taxonomy: "prompt the person to rehearse and repeat the behavior or preparatory behaviors". We used this item to code for skills training
Social support, social change, or supporting structures in the organisation (technique 20)	Social support structures	Abraham 2008 links this constructs to social support theories. It also maps on to social cognitive theories in which reinforcing stimuli from the environment are believed to increase the probability of the desired behaviour.	Item 20 in the taxonomy: "prompting consideration of how others could change their behavior to offer the person help or (instrumental) support, including 'buddy' systems and/or providing social support". We used this item to code for any explicit sign of supporting structures in the organisation or in the social environment of participating providers

#1	[mh "education, continuing"]	1125
#2	[mh "inservice training"]	680
#3	(education* or train) near/6 (conference or course* or lecture* or meeting* or seminar* or session* or symposi* or workshop*):ti,ab	7327
#4	(medical or health* or clinical) next (meeting* or workshop*):ti,ab	68
#5	(educational or learning or training) next (intervention* or program* or strategy or strategies):ti,ab	10637
#6	group next (education or meeting* or seminar* or training):ti,ab	974
#7	(accredit* or credit* or nonaccredit* or noncredit) next (education or conference or course* or meeting* or seminar* or training or workshop*):ti,ab	13
#8	("in service training" or "on the job training"):ti,ab	63
#9	(adult or "case based" or "case method" or "life long" or lifelong or "problem based" or androgogic*) next learning:ti,ab	207
#10	(didactic* or interactive or "inter active") near/3 (educat* or learn* or teach* or train* or course*):ti,ab	757
#11	continuing medical education:ti,ab	242
#12	{or #1‐#11}	18575
#13	[mh "health personnel"]	7122
#14	[mh "health occupations"]	19246
#15	(health* near/2 (personnel or assistant* or professional* or worker*)):ti,ab	4997
#16	(medical or physician* or doctor* or general pract* or family pract* or pharmac* or nurse or nurses or nursing or midwif* or midwiv* or dentist* or therapist* or clinician* or consultant* or team* or staff or GP or pharmacist* or prescriber* or physiotherapist* or therapist*):ti,ab	149563
#17	{or #13‐#16}	164041
#18	#12 and #17	7928
#19	#18 Publication Year from 2013 to 2016	2550

1	exp education, continuing/	57604
2	exp inservice training/	26254
3	((education* or train) adj6 (conference? or course? or lecture? or meeting? or seminar? or session or symposi* or workshop?)).ti,ab.	38058
4	((medical or health* or clinical) adj (meeting? or workshop?)).ti,ab.	688
5	((educational or learning or training) adj (intervention? or program* or strategy or strategies)).ti,ab.	59319
6	(group adj (education or meeting? or seminar? or training)).ti,ab.	3210
7	((accredit* or credit* or nonaccredit* or noncredit*) adj (education or conference? or course? or meeting? or seminar? or training or workshop?)).ti,ab.	315
8	(in service training or on the job training).ti,ab.	1421
9	((adult learning or case based learning or case method learning or life long learning or lifelong learning or problem based learning or androgogic* learning) and (course? or meeting? or seminar? or workshop?)).ti,ab.	1322
10	((didactic* or interactive or inter active) adj3 (educat* or learn* or teach* or train* or course*)).ti,ab.	4425
11	continuing medical education.ti,ab.	4796
12	or/1‐11	170080
13	exp health personnel/	432212
14	exp health occupations/	1539501
15	(health* adj2 (personnel or assistant* or professional* or worker*)).ti,ab.	111133
16	(medical or physician* or doctor* or general pract* or family pract* or pharmac* or nurse or nurses or nursing or midwif* or midwiv* or dentist* or therapist* or clinician* or consultant* or team* or staff or GP or pharmacist* or prescriber* or physiotherapist* or therapist*).ti,ab.	2477782
17	or/13‐16	3764963
18	randomized controlled trial.pt.	434669
19	random*.ti,ab.	895035
20	or/18‐19	1008004
21	animals/	6012972
22	humans/	16431347
23	21 not (21 and 22)	4303731
24	20 not 23	909586
25	12 and 17 and 24	8442
26	"comment on".cm.	687740
27	(systematic review or literature review).ti.	86392
28	(editorial or comment or meta‐analysis or news or review).pt.	3367949
29	"cochrane database of systematic reviews".jn.	16292
30	or/26‐29	3386118
31	25 not 30	7720
32	(2013* or 2014* or 2015* or 2016* or 2017*).dc,dp,ed,ep,yr.	4966061
33	31 and 32	2697

1	*continuing education/	9603
2	exp *medical education/	148744
3	exp *paramedical education/	56716
4	*in service training/	6541
5	*postgraduate education/	4709
6	*adult education/	75
7	((education* or train) adj6 (conference? or course? or lecture? or meeting? or seminar? or session or symposi* or workshop?)).ti,ab.	52879
8	((medical or health* or clinical) adj (meeting? or workshop?)).ti,ab.	815
9	((educational or learning or training) adj (intervention? or program* or strategy or strategies)).ti,ab.	75408
10	(group adj (education or meeting? or seminar? or training)).ti,ab.	4458
11	((accredit* or credit* or nonaccredit* or noncredit*) adj (education or conference? or course? or meeting? or seminar? or training or workshop?)).ti,ab.	456
12	(in service training or on the job training).ti,ab.	1653
13	((adult learning or case based learning or case method learning or life long learning or lifelong learning or problem based learning or androgogic* learning) and (course? or meeting? or seminar? or workshop?)).ti,ab.	1755
14	((didactic* or interactive or inter active) adj3 (educat* or learn* or teach* or train* or course*)).ti,ab.	6399
15	continuing medical education.ti,ab.	6088
16	or/1‐15	327838
17	(health* adj2 (personnel or assistant* or professional* or worker*)).ti,ab.	136749
18	(medical or physician* or doctor* or general pract* or family pract* or pharmac* or nurse or nurses or nursing or midwif* or midwiv* or dentist* or therapist* or clinician* or consultant* or team* or staff or GP or pharmacist* or prescriber* or physiotherapist* or therapist*).ti,ab.	3279277
19	exp *health care personnel/	507604
20	or/17‐19	3538298
21	randomized controlled trial/	459863
22	random*.ti,ab.	1147320
23	controlled trial.ti,ab.	116334
24	or/21‐23	1248814
25	16 and 20 and 24	11423
26	(systematic review or literature review).ti.	102008
27	"cochrane database of systematic reviews".jn.	5034
28	26 or 27	107033
29	25 not 28	11076
30	(2013* or 2014* or 2015* or 2016* or 2017*).dd.	3315934
31	29 and 30	2462
32	limit 31 to embase	1568

S1	("continuing education" or "mandatory continuing education" or "professional continuing education" or "workshops" or "conferences gatherings" or "meetings" or "seminars" or "courses" or "advanced courses" or "methods courses" or "minicourses" or "noncredit courses" or "practicums" or "refresher courses" or "required courses" or "united states government course" or "education courses" or "internship programs" or "on the job training") AND ("medical education")	1,945
S2	((medical or health* or clinical) N0 (course* or meeting* or seminar* or workshop*))	460
S3	education* N0 (conference* or course* or intervention* or lecture* or meeting* or program* or seminar* or session* or symposi* or workshop*)	86,118
S4	group N0 (education or meeting* or seminar* or training)	2,645
S5	(accredit* or credit* or nonaccredit* or noncredit) N0 (education or conference or course* or meeting* or seminar* or training or workshop)	3,181
S6	("job training" or internship program*)	19,512
S7	medical or health* or physician* or doctor* or general pract* or family pract* or pharmac* or nurse or nurses or midwif* or midwiv* or dentist* or therapist*	160,244
S8	S3 OR S4 OR S5 OR S6	108,064
S9	S7 AND S8	14,430
S10	S1 OR S2 OR S9	16,334
S11	(randomly or randomised or randomized or "controlled trial")	16,923
S12	S10 AND S11	309
S13	S12	46
S14	("continuing education" or "mandatory continuing education" or "professional continuing education" or "workshops" or "conferences gatherings" or "meetings" or "seminars" or "courses" or "advanced courses" or "methods courses" or "minicourses" or "noncredit courses" or "practicums" or "refresher courses" or "required courses" or "united states government course" or "education courses" or "internship programs" or "on the job training") AND ("medical education")	1,945
S15	((medical or health* or clinical) N0 (course* or meeting* or seminar* or workshop*))	460
S16	education* N0 (conference* or course* or intervention* or lecture* or meeting* or program* or seminar* or session* or symposi* or workshop*)	86,118
S17	group N0 (education or meeting* or seminar* or training)	2,645
S18	(accredit* or credit* or nonaccredit* or noncredit) N0 (education or conference or course* or meeting* or seminar* or training or workshop)	3,181
S19	("job training" or internship program*)	19,512
S20	medical or health* or physician* or doctor* or general pract* or family pract* or pharmac* or nurse or nurses or midwif* or midwiv* or dentist* or therapist*	160,244
S21	S16 OR S17 OR S18 OR S19	108,064
S22	S20 AND S21	14,430
S23	S14 OR S15 OR S22	16,334
S24	(randomly or randomised or randomized or "controlled trial")	16,923
S25	S23 AND S24	309
S26	S25 Limiters ‐ Date Published: 20130101‐20161231	46

# 1	4,987	TS=("medical course" or "medical courses" or "medical meeting" or "medical meetings" or "medical workshop" or "medical workshops" or "clinical workshop" or "clinical workshops" or "continuing medical education") Indexes=SCI‐EXPANDED, SSCI Timespan=All years
# 2	528	TS=("educational meeting" or "educational meetings" or "educational workshop" or "educational workshops" or "educational course" or "educational courses") and TS=(medical or health* or physician* or doctor* or "general practitioner" or "general practitioners" or "family practitioner" or "family practitioners" or pharmacist* or nurse or nurses or midwif* or midwiv* or dentist* or therapist) Indexes=SCI‐EXPANDED, SSCI Timespan=All years*
# 3	1,046,804	TS=(randomly or "random allocation" or randomised or randomized or "conrtolled trial" or "control group") Indexes=SCI‐EXPANDED, SSCI Timespan=All years
# 4	5,492	#2 OR #1 Indexes=SCI‐EXPANDED, SSCI Timespan=All years
# 5	714	#4 AND #3 Indexes=SCI‐EXPANDED, SSCI Timespan=All years
# 6	175	#5 Indexes=SCI‐EXPANDED, SSCI Timespan=2013‐2016

#1	MeSH descriptor: [Education, Continuing] explode all trees	874
#2	MeSH descriptor: [Inservice Training] explode all trees	467
#3	(education* or train) near/6 (conference or course* or lecture* or meeting* or seminar* or session* or symposi* or workshop*):ti,ab	3709
#4	(medical or health* or clinical) next (meeting* or workshop*):ti,ab	48
#5	(educational or learning or training) next (intervention* or program* or strategy or strategies):ti,ab	6148
#6	group next (education or meeting* or seminar* or training):ti,ab	849
#7	(accredit* or credit* or nonaccredit* or noncredit) next (education or conference or course* or meeting* or seminar* or training or workshop*):ti,ab	6
#8	("in service training" or "on the job training"):ti,ab	46
#9	(adult or "case based" or "case method" or "life long" or lifelong or "problem based" or androgogic*) next learning:ti,ab	136
#10	(didactic* or interactive or "inter active") near/3 (educat* or learn* or teach* or train*):ti,ab	395
#11	"continuing medical education":ti,ab	186
#12	#1 or #2 or #3 or #4 or #5 or #6 or #7 or #8 or #9 or #10 or #11	10670
#13	sr‐epoc	7121
#14	#12 and #13 in Trials	973

#	Searches	Results
1	exp *Education, Continuing/	0
2	exp *Inservice Training/	0
3	((education* or train) adj6 (conference? or course? or lecture? or meeting? or seminar? or session or symposi* or workshop?)).ti,ab.	1729
4	((medical or health* or clinical) adj (meeting? or workshop?)).ti,ab.	77
5	((educational or learning or training) adj (intervention? or program* or strategy or strategies)).ti,ab.	2527
6	(group adj (education or meeting? or seminar? or training)).ti,ab.	150
7	((accredit* or credit* or nonaccredit* or noncredit*) adj (education or conference? or course? or meeting? or seminar? or training or workshop?)).ti,ab.	10
8	(in service training or on the job training).ti,ab.	55
9	((adult learning or case based learning or case method learning or life long learning or lifelong learning or problem based learning or androgogic* learning) and (course? or meeting? or seminar? or workshop?)).ti,ab.	65
10	((didactic* or interactive or inter active) adj3 (educat* or learn* or teach* or train*)).ti,ab.	187
11	continuing medical education.ti,ab.	240
12	or/1‐11	4543
13	randomized controlled trial.pt.	812
14	random*.ti,ab.	46773
15	or/13‐14	46974
16	Animals/	16
17	Humans/	47
18	16 not (16 and 17)	2
19	15 not 18	46974
20	12 and 19	688
21	"comment on".cm.	35293
22	(systematic review or literature review).ti.	4327
23	(editorial or comment or meta‐analysis or news or review).pt.	53717
24	"cochrane database of systematic reviews".jn.	121
25	or/21‐24	57948
26	20 not 25	647

S1	ALL("continuing education" or "mandatory continuing education" or "professional continuing education" or "workshops" or "conferences gatherings" or "meetings" or "seminars" or "courses" or "advanced courses" or "methods courses" or "minicourses" or "noncredit courses" or "practicums" or "refresher courses" or "required courses" or "united states government course" or "education courses" or "internship programs" or "on the job training") AND ALL("medical education")	1787
S2	ALL(medical P/0 course* or health* P/0 course* or clinical P/0 course* or medical P/0 meeting* or health* P/0 meeting* or clinical P/0 meeting* or medical P/0 seminar* or health* P/0 seminar* or clinical P/0 seminar* or medical P/0 workshop* or health* P/0 workshop* or clinical P/0 workshop*)	332
S3	ALL(educational P/0 conference* or educational P/0 course* or educational P/0 intervention* or educational P/0 lecture* or educational P/0 meeting* or educational P/0 program* or educational P/0 seminar* or educational P/0 session* or educational P/0 symposi* or educational P/0 workshop) AND ALL(medical or health or physician* or doctor* or general P/0 pract* or family P/0 pract* or pharmac* or nurse or nurses or midwif* or midwiv* or dentist* or therapist*)	3547
S4	ALL(group P/0 education or group P/0 meeting* or group P/0 seminar* or group P/0 training) AND ALL(medical or health* or physician* or doctor* or general P/0 pract* or family P/0 pract* or pharmac* or nurse or nurses or midwif* or midwiv* or dentist* or therapist*)	285
S5	ALL(accredit* P/0 education or credit* P/0 education or nonaccredit* P/0 education or noncredit* P/0 education or accredit* P/0 conference* or credit* P/0 conference* or nonaccredit* P/0 conference* or noncredit* P/0 conference* or accredit* P/0 course* or credit* P/0 course* or nonaccredit* P/0 course* or noncredit* P/0 course* or accredit* P/0 meeting* or credit* P/0 meeting* or nonaccredit* P/0 meeting* or noncredit* P/0 meeting* or accredit* P/0 seminar* or credit* P/0 seminar* or nonaccredit* P/0 seminar* or noncredit* P/0 seminar* or accredit* P/0 training or credit* P/0 training or nonaccredit* P/0 training or noncredit* P/0 training or accredit* P/0 workshop or credit* P/0 workshop or nonaccredit* P/0 workshop or noncredit* P/0 workshop) AND ALL(medical or health* or physician* or doctor* or general P/0 pract* or family P/0 pract* or pharmac* or nurse or nurses or midwif* or midwiv* or dentist* or therapist*)	255
S6	ALL("job training" or internship P/0 program) AND ALL(medical or health or physician* or doctor* or general P/0 pract* or family P/0 pract* or pharmac* or nurse or nurses or midwif* or midwiv* or dentist* or therapist*)	2189
S7	(ALL("continuing education" OR "mandatory continuing education" OR "professional continuing education" OR "workshops" OR "conferences gatherings" OR "meetings" OR "seminars" OR "courses" OR "advanced courses" OR "methods courses" OR "minicourses" OR "noncredit courses" OR "practicums" OR "refresher courses" OR "required courses" OR "united states government course" OR "education courses" OR "internship programs" OR "on the job training") AND ALL("medical education")) OR ALL(medical PRE/0 course* OR health* PRE/0 course* OR clinical PRE/0 course* OR medical PRE/0 meeting* OR health* PRE/0 meeting* OR clinical PRE/0 meeting* OR medical PRE/0 seminar* OR health* PRE/0 seminar* OR clinical PRE/0 seminar* OR medical PRE/0 workshop* OR health* PRE/0 workshop* OR clinical PRE/0 workshop) OR (ALL(educational PRE/0 conference OR educational PRE/0 course* OR educational PRE/0 intervention* OR educational PRE/0 lecture* OR educational PRE/0 meeting* OR educational PRE/0 program* OR educational PRE/0 seminar* OR educational PRE/0 session* OR educational PRE/0 symposi* OR educational PRE/0 workshop) AND ALL(medical OR health OR physician* OR doctor* OR general PRE/0 pract* OR family PRE/0 pract* OR pharmac* OR nurse OR nurses OR midwif* OR midwiv* OR dentist* OR therapist)) OR (ALL(group PRE/0 education OR group PRE/0 meeting OR group PRE/0 seminar* OR group PRE/0 training) AND ALL(medical OR health* OR physician* OR doctor* OR general PRE/0 pract* OR family PRE/0 pract* OR pharmac* OR nurse OR nurses OR midwif* OR midwiv* OR dentist* OR therapist)) OR (ALL(accredit PRE/0 education OR credit* PRE/0 education OR nonaccredit* PRE/0 education OR noncredit* PRE/0 education OR accredit* PRE/0 conference* OR credit* PRE/0 conference* OR nonaccredit* PRE/0 conference* OR noncredit* PRE/0 conference* OR accredit* PRE/0 course* OR credit* PRE/0 course* OR nonaccredit* PRE/0 course* OR noncredit* PRE/0 course* OR accredit* PRE/0 meeting* OR credit* PRE/0 meeting* OR nonaccredit* PRE/0 meeting* OR noncredit* PRE/0 meeting* OR accredit* PRE/0 seminar* OR credit* PRE/0 seminar* OR nonaccredit* PRE/0 seminar* OR noncredit* PRE/0 seminar* OR accredit* PRE/0 training OR credit* PRE/0 training OR nonaccredit* PRE/0 training OR noncredit* PRE/0 training OR accredit* PRE/0 workshop OR credit* PRE/0 workshop OR nonaccredit* PRE/0 workshop OR noncredit* PRE/0 workshop) AND ALL(medical OR health* OR physician* OR doctor* OR general PRE/0 pract* OR family PRE/0 pract* OR pharmac* OR nurse OR nurses OR midwif* OR midwiv* OR dentist* OR therapist)) OR (ALL("job training" OR internship PRE/0 program) AND ALL(medical OR health* OR physician* OR doctor* OR general PRE/0 pract* OR family PRE/0 pract* OR pharmac* OR nurse OR nurses OR midwif* OR midwiv* OR dentist* OR therapist*))	7970
S8	ALL(randomly or randomised or randomized or "controlled trial")	15186
S9	((ALL("continuing education" OR "mandatory continuing education" OR "professional continuing education" OR "workshops" OR "conferences gatherings" OR "meetings" OR "seminars" OR "courses" OR "advanced courses" OR "methods courses" OR "minicourses" OR "noncredit courses" OR "practicums" OR "refresher courses" OR "required courses" OR "united states government course" OR "education courses" OR "internship programs" OR "on the job training") AND ALL("medical education")) OR ALL(medical PRE/0 course* OR health* PRE/0 course* OR clinical PRE/0 course* OR medical PRE/0 meeting* OR health* PRE/0 meeting* OR clinical PRE/0 meeting* OR medical PRE/0 seminar* OR health* PRE/0 seminar* OR clinical PRE/0 seminar* OR medical PRE/0 workshop* OR health* PRE/0 workshop* OR clinical PRE/0 workshop) OR (ALL(educational PRE/0 conference OR educational PRE/0 course* OR educational PRE/0 intervention* OR educational PRE/0 lecture* OR educational PRE/0 meeting* OR educational PRE/0 program* OR educational PRE/0 seminar* OR educational PRE/0 session* OR educational PRE/0 symposi* OR educational PRE/0 workshop) AND ALL(medical OR health OR physician* OR doctor* OR general PRE/0 pract* OR family PRE/0 pract* OR pharmac* OR nurse OR nurses OR midwif* OR midwiv* OR dentist* OR therapist)) OR (ALL(group PRE/0 education OR group PRE/0 meeting OR group PRE/0 seminar* OR group PRE/0 training) AND ALL(medical OR health* OR physician* OR doctor* OR general PRE/0 pract* OR family PRE/0 pract* OR pharmac* OR nurse OR nurses OR midwif* OR midwiv* OR dentist* OR therapist)) OR (ALL(accredit PRE/0 education OR credit* PRE/0 education OR nonaccredit* PRE/0 education OR noncredit* PRE/0 education OR accredit* PRE/0 conference* OR credit* PRE/0 conference* OR nonaccredit* PRE/0 conference* OR noncredit* PRE/0 conference* OR accredit* PRE/0 course* OR credit* PRE/0 course* OR nonaccredit* PRE/0 course* OR noncredit* PRE/0 course* OR accredit* PRE/0 meeting* OR credit* PRE/0 meeting* OR nonaccredit* PRE/0 meeting* OR noncredit* PRE/0 meeting* OR accredit* PRE/0 seminar* OR credit* PRE/0 seminar* OR nonaccredit* PRE/0 seminar* OR noncredit* PRE/0 seminar* OR accredit* PRE/0 training OR credit* PRE/0 training OR nonaccredit* PRE/0 training OR noncredit* PRE/0 training OR accredit* PRE/0 workshop OR credit* PRE/0 workshop OR nonaccredit* PRE/0 workshop OR noncredit* PRE/0 workshop) AND ALL(medical OR health* OR physician* OR doctor* OR general PRE/0 pract* OR family PRE/0 pract* OR pharmac* OR nurse OR nurses OR midwif* OR midwiv* OR dentist* OR therapist)) OR (ALL("job training" OR internship PRE/0 program) AND ALL(medical OR health* OR physician* OR doctor* OR general PRE/0 pract* OR family PRE/0 pract* OR pharmac* OR nurse OR nurses OR midwif* OR midwiv* OR dentist* OR therapist*))) AND ALL(randomly OR randomised OR randomized OR "controlled trial")	144

# 6	77	#5 AND #4 Databases=SCI‐EXPANDED, SSCI Timespan=2011‐01‐01 ‐ 2013‐02‐12
# 5	4,233	#1 OR #2 OR #3 Databases=SCI‐EXPANDED, SSCI Timespan=All Years
# 4	739,403	TS=(randomly or "random allocation" or randomised or randomized or "conrtolled trial" or "control group") Databases=SCI‐EXPANDED, SSCI Timespan=All Years
# 3	471	TS=("medical course" or "medical courses" or "medical meeting" or "medical meetings" or "medical workshop" or "medical workshops" or "clinical workshop" or "clinical workshops") Databases=SCI‐EXPANDED, SSCI Timespan=All Years
# 2	3,437	TS=("continuing medical education") Databases=SCI‐EXPANDED, SSCI Timespan=All Years
# 1	345	TS=("educational meeting" or "educational meetings" or "educational workshop" or "educational workshops" or "educational course" or "educational courses") and TS=(medical or health* or physician* or doctor* or "general practitioner" or "general practitioners" or "family practitioner" or "family practitioners" or pharmacist* or nurse or nurses or midwif* or midwiv* or dentist* or therapist*) Databases=SCI‐EXPANDED, SSCI Timespan=All Years

*Study characteristics*
Methods	Design: cluster RCT
Participants	Country: Turkey Setting: primary care Profession: GPs Number of health professionals: 25 Number of patients: 641 baseline, 376 follow‐up Type of targeted behaviour: prescribing Proportion of eligible providers who participated: not clear
Interventions	• CME: 3‐day training programme on rational use of drugs. Designed according to "problem‐based Groningen/WHO model" Interactive training methods: small‐group discussions, role‐playing, patient simulations, and objective structured clinical examination and training in designing a treatment for selected indications • Control: no intervention Comparison 1: 1 vs 2
Outcomes	Providers: score for choosing rational drugs Patients: none
Notes	Number of behaviour change techniques: 3 Additional material to take home: no Duration of educational meetings: 24 hours Intervention fidelity: proportion of attendance: 100% Use of theory: problem‐based Interactive vs didactic educational meetings: interactive Complexity of targeted behaviour: high Importance of outcomes: low Time to follow‐up: 4 months Decrease or increase in behaviour: other improvements Type of teacher: researcher Dates when study was conducted: 1999 to 2000 Funding: not reported, but the study was conducted at the Marmara University School of Medicine, Turkey Declaration of interest: no information
*Risk of bias*
Bias	Authors' judgement	Support for judgement
Random sequence generation (selection bias)	Unclear risk	"They were randomly allocated ..."
Allocation concealment (selection bias)	Unclear risk	See above
Recruiters blinded or recruitment taking place before randomisation	Unclear risk	No information
Baseline (provider) characteristics similar	Low risk	"No statistical significance observed between the groups..."
Baseline (patient) characteristics similar	Low risk	Table 5
Baseline outcome measurement similar	Low risk	Table 4: choosing rational drug(s)
Blinding of outcome assessment (detection bias) All outcomes	Low risk	"All the questionnaires were numbered and the origin of the GP and [of the] patient was masked [for both] prescriptions and questionnaires to keep the assessors blind"
Adequately protected against contamination	Unclear risk	Not clear whether some GPs worked at the same healthcare centres
Incomplete outcome data (attrition bias) All outcomes	Unclear risk	No information
Selective reporting (reporting bias)	Low risk	No reason to suspect
Other bias	Low risk	None identified
Risk of bias overall	Unclear risk	Unclear risk overall

*Study characteristics*
Methods	Design: cluster RCT; randomised by provider
Participants	Country: Denmark Setting: paediatric outpatient clinics Profession: Medical doctors and nurses Number of health professionals: 30 Number of patients: 764 Type of targeted behaviour: communication behaviour Proportion of eligible providers who participated: not reported
Interventions	• CME: 5‐day communication course described by Mcguire founded on social learning. Comprised 2 blocks of 2 days and 3 days, and a period of 4 weeks separated the 2 • Control: no intervention Comparison 1: 1 vs 2
Outcomes	Providers: % of patients answering 'satisfied' Patients: none
Notes	Number of behaviour change techniques: 4 Additional material to take home: none Duration of educational meetings: 40 hours Intervention fidelity: proportion of attendance: 92% Use of theory: social learning theory Interactive vs didactic educational meetings: interactive Complexity of targeted behaviour: low Importance of outcomes: high Time to follow‐up: 5 months Decrease or increase in behaviour: other improvements Type of teacher: professional colleague from same or other setting (paediatrician trained as a teacher) Dates when study was conducted: first phase (pre‐course sample): 16 August 2004 to June 2005 Funding: supported by County of Vejle Denmark, Fredericia and Kolding Hospitals, Center for Internal Development & Education, Danish Nurses Organisation Declaration of interest: not reported
*Risk of bias*
Bias	Authors' judgement	Support for judgement
Random sequence generation (selection bias)	Unclear risk	No mention of how randomisation was conducted, or whether any software systems were used
Allocation concealment (selection bias)	Unclear risk	No mention of software systems used or whether sealed envelopes were used
Recruiters blinded or recruitment taking place before randomisation	Unclear risk	No information
Baseline (provider) characteristics similar	Low risk	Only 1 provider
Baseline (patient) characteristics similar	Low risk	Only parents were included
Baseline outcome measurement similar	High risk	Outcomes were not similar
Blinding of outcome assessment (detection bias) All outcomes	Unclear risk	No information
Adequately protected against contamination	Unclear risk	Yes, adequately protected
Incomplete outcome data (attrition bias) All outcomes	Low risk	All outcome data protected
Selective reporting (reporting bias)	Low risk	All outcomes are reported and are very clearly presented
Other bias	Low risk	All outcomes are reported and any risks made explicit
Risk of bias overall	Unclear risk	Unclear risk overall

*Study characteristics*
Methods	Design: cluster randomised trial
Participants	Country: France Setting: primary care Profession: general practitioners Number of health professionals: 45 Number of patients: 3076 Type of targeted behaviour: colorectal cancer screening, communication skills Proportion of eligible providers who participated: 17%
Interventions	• CME: 4‐hour training course focusing on patient‐centred care, videos, role‐playing, and interactive discussion + a memo summarising the main communication skills • Control: no training Comparison 1: 1 vs 2
Outcomes	Providers: % of patient participation per GP Patients: none
Notes	Number of behaviour change techniques: 3 Additional material to take home: yes Duration of educational meetings: 4 hours Intervention fidelity: proportion of attendance: 36% Use of theory: no theory Interactive vs didactic educational meetings: interactive Complexity of targeted behaviour: low Importance of outcomes: high Time to follow‐up: 7 months Decrease or increase in behaviour: increase Type of teacher: researcher Dates when study was conducted: 10 December 2011 to 9 July 2012 Funding: French local mass screening organisation Prevention Sante Val d’Oise (PSVO) for education courses and data collection Declaration of interest: "the authors declare that they have no competing interests"
*Risk of bias*
Bias	Authors' judgement	Support for judgement
Random sequence generation (selection bias)	Low risk	"an independent biostatistician randomised 50 practices per arm in October 2011 based on a computer‐generated randomisation list".
Allocation concealment (selection bias)	Low risk	See above
Recruiters blinded or recruitment taking place before randomisation	Unclear risk	Figure 1: recruited after randomisation; blinding unclear
Baseline (provider) characteristics similar	Low risk	Table 1
Baseline (patient) characteristics similar	Unclear risk	Not applicable
Baseline outcome measurement similar	Low risk	"Of note, the patient participation rate per GP over the 2 years before the study period (from December 10, 2009 to December 9, 2011) was 24.3 (9.1) in the intervention group and 24.7 (9.5) in the control group (P = 0.8)"
Blinding of outcome assessment (detection bias) All outcomes	Low risk	"outcome assessors remained blind to the allocation"
Adequately protected against contamination	Low risk	Clusters (practices)
Incomplete outcome data (attrition bias) All outcomes	High risk	Figure 1: lost many practices
Selective reporting (reporting bias)	Low risk	No reason to suspect
Other bias	Low risk	None identified
Risk of bias overall	High risk	High risk of bias overall

*Study characteristics*
Methods	Design: cluster RCT; randomised by practice
Participants	Country: The Netherlands Setting: community‐based care Profession: physiotherapists Number of health professionals: 113/68 practices Number of patients: 500 Type of targeted behaviour: general management of a problem (low back pain) Proportion of eligible providers who participated: 21%
Interventions	• CME: 2.5 hours × 2 didactic and interactive workshops targeted at barriers + feedback + reminders • No intervention control (guidelines by mail)
Outcomes	Professional practice: proportion of adherence to guidelines for 4 recommendations for low back pain Patients: % of patients at sick leave during previous 6 weeks at 52 weeks
Notes	Number of behaviour change techniques: 3 Additional material to take home: no Duration of educational meetings: 5 hours Intervention fidelity: proportion of attendance: 75% Use of theory: no Interactive vs didactic educational meetings: interactive Complexity of targeted behaviour: low Importance of outcomes: low Time to follow‐up: 12 months Decrease or increase in behaviour: other improvement Type of teacher: researcher Dates when study was conducted: May 2001 to December 2002 Funding: Ministry of Health, Welfare and Sports Declaration of interest: no information
*Risk of bias*
Bias	Authors' judgement	Support for judgement
Random sequence generation (selection bias)	Low risk	"A statistician, who was not involved in this trial, drew up an allocation schedule using a computerised random number generator"
Allocation concealment (selection bias)	Low risk	"The primary investigator (GEB), without any knowledge of the practices, listed them alphabetically according to the name of their street address, and subsequently assigned them to the intervention or control group using the allocation schedule"
Recruiters blinded or recruitment taking place before randomisation	Unclear risk	"All participating physiotherapists were asked to include a maximum of 10 consecutive patients who were (for the first time or again) referred for physiotherapy for a new episode of non‐specific low back pain"
Baseline (provider) characteristics similar	Unclear risk	"Physiotherapists in the intervention group were older than those in the control group (p = 0.011)" Also more physiotherapists in the control group (6,8%) had a postgraduate education on chronic pain, and more of those in the intervention group had education on low back pain (75% vs 69,5)
Baseline (patient) characteristics similar	Low risk	"There were no differences in patient characteristics between the two groups (table 2)"
Baseline outcome measurement similar	Unclear risk	Not measured
Blinding of outcome assessment (detection bias) All outcomes	Low risk	"Two reviewers independently assessed the registration form using the algorithm without being aware of the group allocation"
Adequately protected against contamination	Low risk	Randomised by practice
Incomplete outcome data (attrition bias) All outcomes	Unclear risk	Lost 25% of physiotherapists; no information on how many clusters (practices) were lost
Selective reporting (reporting bias)	Low risk	No reason to suspect
Other bias	Low risk	None identified
Risk of bias overall	Unclear risk	Unclear risk overall

*Study characteristics*
Methods	Design: cluster RCT; randomised by pharmacy
Participants	Country: Australia Setting: community‐based care Profession: pharmacists Number of health professionals: 90 pharmacies Number of patients: 87,130 prescriptions Type of targeted behaviour: clinical pharmacy Proportion of eligible providers who participated: not reported
Interventions	• CME: 1‐week basic training course: lectures, case studies, and hospital visits + manual + guidelines + monthly index of medical specialities • Control: no intervention Comparison 1: 1 vs 2
Outcomes	Providers: % of clinical interventions on prescriptions Patients: none
Notes	Number of behaviour change techniques: 1 Additional material to take home: manual, guidelines, and index Duration of educational meetings: 40 hours Intervention fidelity: proportion of attendance: not assessable Use of theory: no theory Interactive vs didactic educational meetings: didactic Complexity of targeted behaviour: high Importance of outcomes: low Time to follow‐up: 2 weeks Decrease or increase in behaviour: increase Type of teacher: professional colleague from other setting Dates when study was conducted: no information Funding: Commonwealth Department of Human Services and Health Declaration of interest: no information
*Risk of bias*
Bias	Authors' judgement	Support for judgement
Random sequence generation (selection bias)	Unclear risk	"Three groups were randomly selected … and a fourth group was conveniently sampled"
Allocation concealment (selection bias)	Unclear risk	No information on allocation concealment
Recruiters blinded or recruitment taking place before randomisation	High risk	Recruitment took place before randomisation (see Figure 1), but "the recruitment rates differed markedly between the four groups" and "the lower recruitment rate in group B suggests that it may have been a self selected rather than random sample"
Baseline (provider) characteristics similar	Unclear risk	Provider baseline characteristics not reported
Baseline (patient) characteristics similar	Unclear risk	Baseline characteristics for patients not reported
Baseline outcome measurement similar	Low risk	Rates of interventions were reported at baseline and were significantly different between groups but were adjusted in the analyses
Blinding of outcome assessment (detection bias) All outcomes	Unclear risk	Pharmacists (providers) recorded outcomes; some were "double‐screened by a community pharmacist"
Adequately protected against contamination	Low risk	Randomised by pharmacy
Incomplete outcome data (attrition bias) All outcomes	Low risk	Appears there are no missing data
Selective reporting (reporting bias)	Low risk	No reason to suspect reporting bias; all relevant outcomes in the methods section were reported in the results section
Other bias	Unclear risk	Poor documentation during the study was reported
Risk of bias overall	High risk	High risk of bias overall

*Study characteristics*
Methods	Design: cluster RCT; randomised by hospital
Participants	Country: Sør‐Afrika Setting: hospital Profession: managers, doctors, midwives, nurses, dieticians, occupational therapists, speech therapists, physiotherapists, social workers Number of health professionals: 324 Number of patients: not clear Type of targeted behaviour: preventive care Proportion of eligible providers who participated: 100%
Interventions	All had introductory workshop • CME: 'on‐site' facilitation (Group A) entailed 2 site visits to hospitals, each lasting 2 to 3 hours • Control: 'off‐site': 1‐ or 2‐day, 'hands‐on' training workshop at hospitals identified as centres of excellence. This took place 6 to 8 weeks after introductory workshop Comparison 4: 2 vs 1 (outreach vs educational meeting)
Outcomes	Providers: implementation score Patients: none
Notes	Number of behaviour change techniques: 2 (goal‐setting + education) Additional material to take home: no Duration of educational meetings: 4 hours Intervention fidelity: proportion of attendance: not clear Use of theory: no theory Interactive vs didactic educational meetings: interactive Complexity of targeted behaviour: low Importance of outcomes: high Time to follow‐up: 8 months Decrease or increase in behaviour: other improvements Type of teacher: others: centre of excellence Dates when study was conducted: "they launched the Fara Ngwana ('hold the baby') outreach in August 2003. In the Mpumalanga Province the Ukubamba Umtwana Kuwe ('hold the baby tightly') outreach, launched in March 2004, was the responsibility of the Subdirectorate: Nutrition of the Department of Health and Social Services and was one of the priority programmes of the Integrated Nutrition Programme. In Gauteng seven hospitals were targeted for implementation support in 2003 and another five in 2005. In Mpumalanga seven hospitals were targeted for 2004, 11 for 2005 and eight for 2006. All the hospitals in the trial were state‐run, public hospitals" Funding: "this project was funded by the Mpumalanga Department of Health and Social Services, the Gauteng Department of Health, the MRC Research Unit for Maternal and Infant Health Care Strategies and the Italian Cooperation" Declaration of interest: "the authors declare that they have no competing interests"
*Risk of bias*
Bias	Authors' judgement	Support for judgement
Random sequence generation (selection bias)	Low risk	"One hospital in each pair was randomly allocated to Group A, the other to Group B, by spinning a coin. Group A received on‐site facilitation and Group B off‐site facilitation"
Allocation concealment (selection bias)	Unclear risk	See above
Recruiters blinded or recruitment taking place before randomisation	Unclear risk	Not reported
Baseline (provider) characteristics similar	Unclear risk	Not reported
Baseline (patient) characteristics similar	Unclear risk	Not reported
Baseline outcome measurement similar	Unclear risk	No baseline measurements
Blinding of outcome assessment (detection bias) All outcomes	Unclear risk	Not clear
Adequately protected against contamination	Unclear risk	"... the same respected resource persons were responsible for the facilitation at, interaction with, and feedback to all hospitals..."
Incomplete outcome data (attrition bias) All outcomes	Low risk	No clusters lost
Selective reporting (reporting bias)	Low risk	No reason to suspect
Other bias	Low risk	None identified
Risk of bias overall	Unclear risk	Unclear risk of bias overall

PERMALINK

Continuing education meetings and workshops: effects on professional practice and healthcare outcomes

Louise Forsetlund

Mary Ann O'Brien

Lisa Forsén

Leah Mwai

Liv Merete Reinar

Mbah P Okwen

Tanya Horsley

Christopher J Rose

Abstract

Background

Objectives

Search methods

Selection criteria

Data collection and analysis

Main results

Authors' conclusions

Plain language summary

Summary of findings

Background

Description of the condition

Description of the intervention

Are educational meetings and workshops, alone or supported by other interventions, effective in improving professional practice or healthcare outcomes?

How does the effectiveness of educational meetings compare with that of other interventions?

Can educational meetings be made more effective by modifying how they are done?

How the intervention might work

1. Behaviour change techniques used in the included studies.

2. Targeted behaviours in the included studies.

Why it is important to do this review

Objectives

Methods

Criteria for considering studies for this review

Types of studies

Types of participants

Types of interventions

Types of outcome measures

Search methods for identification of studies

Electronic searches

Searching other resources

Data collection and analysis

Selection of studies

Data extraction and management

Assessment of risk of bias in included studies

Measures of treatment effect

Unit of analysis issues

Cluster randomised trials

Studies with more than two arms

Dealing with missing data

Assessment of heterogeneity

Assessment of reporting biases

Data synthesis

Subgroup analysis and investigation of heterogeneity

Sensitivity analysis

Summary of findings and assessment of the certainty of the evidence

Results

Description of studies

Results of the search

1.

Included studies

Characteristics of providers and settings

Targeted behaviours

Characteristics of the intervention

Outcome measures

Comparisons

Excluded studies

Risk of bias in included studies

2.

3.

Random sequence generation

Concealment of allocation

Recruitment of participants

Similarity of provider baseline characteristics

Similarity of patient baseline characteristics

Similarity of baseline outcome measures

Blinding of outcome assessment

Protection against contamination

Incomplete outcome data (attrition bias)

Selective reporting (reporting bias)

Other potential sources of bias

*Study characteristics*
Methods	Cluster RCT
Participants	Country: Zambia Setting: community‐based care Profession: prescribers (clinical officers and medical officers) at 16 health centres Number of health professionals: 52 Number of patients: 5685 Type of targeted behaviour: general management of a problem Proportion of eligible providers who participated: 84%
Interventions	• CME: 3‐day interactive seminar × 2 • Control: no intervention Comparison 1: 1 vs 2
Outcomes	Professional practice: overall proportion of patients adequately managed Patients: none
Notes	Number of behaviour change techniques: 1 Additional material to take home: no Duration of educational meetings: 48 hours Intervention fidelity: proportion of attendance: 85% Use of theory: no Interactive vs didactic educational meetings: interactive Complexity of targeted behaviour: low Importance of outcomes: low Time to follow‐up: 3 months Decrease or increase in behaviour: other kind of improvement Type of teacher: researcher Dates when study was conducted: "January to September 1991" Funding: not reported, but the study was "a collaborative project between policy makers (the Ministry of Health), researchers, and prescribers" Declaration of interest: not clear
*Risk of bias*
Bias	Authors' judgement	Support for judgement
Random sequence generation (selection bias)	Unclear risk	"A randomized controlled trial with the health center as study unit was carried out ... 16 health centers were pair matched for geographic location, number of prescribers, and average number of outpatients per month. Following the collection of baseline data, one health center of each pair was randomly allocated to the intervention group and the other to the control group"
Allocation concealment (selection bias)	Unclear risk	Not clear
Recruiters blinded or recruitment taking place before randomisation	Unclear risk	Not reported
Baseline (provider) characteristics similar	Low risk	Groups similar (table 1)
Baseline (patient) characteristics similar	Low risk	Groups similar (table 1)
Baseline outcome measurement similar	Low risk	Groups similar (table 2)
Blinding of outcome assessment (detection bias) All outcomes	Low risk	"...the evaluation team did not know whether a patient card came from an intervention or a control health center"
Adequately protected against contamination	Low risk	Cluster trial
Incomplete outcome data (attrition bias) All outcomes	Unclear risk	Not clear whether any clusters were lost
Selective reporting (reporting bias)	Low risk	No reason to suspect
Other bias	Low risk	None identified
Risk of bias overall	Unclear risk	Unclear risk of bias overall

*Study characteristics*
Methods	Design: cluster randomised trial; randomised by provider
Participants	Country: USA Setting: community health centres Profession: family medicine clinicians Number of health professionals: 13 Number of patients: 319 Type of targeted behaviour: physical activity counselling Proportion of eligible providers who participated: 87%
Interventions	• CME: 1 hour * 4 training sessions: techniques known to be effective for communication training (didactic materials, skills and competency checklists, role‐play, and cognitive rehearsal. Introduction to the % As program (the Health Living Program), interactive, each clinician received a competency checklist completed by a standardised patient. No booster sessions; no follow‐up training • Control: not reported, but waiting list Comparison 1: 1 vs 2
Outcomes	Providers: PAEI (physical activity exit interview) score after patients' office visit, score range 1 to 15, higher better Patients: none
Notes	Number of behaviour change techniques: 2 Additional material to take home: no Duration of educational meetings: 4 hours Intervention fidelity: proportion of attendance: not reported Use of theory: no Interactive vs didactic educational meetings: interactive Complexity of targeted behaviour: low Importance of outcomes: low Time to follow‐up: 6 months Decrease or increase in behaviour: increase Type of teacher: researcher Dates when study was conducted: June 2009 to October 2011 Funding: National Cancer Institute of the National Institutes of Health Declaration of interest: not reported in article
*Risk of bias*
Bias	Authors' judgement	Support for judgement
Random sequence generation (selection bias)	Unclear risk	"Family medicine clinicians at 2 community health centers were randomized to Group 1 or Group 2 intervention"
Allocation concealment (selection bias)	Unclear risk	See above
Recruiters blinded or recruitment taking place before randomisation	Unclear risk	Seems like clinicians were recruited before randomisation ("clinicians were recruited, enrolled, and randomized ..."), but unclear if nurse assistants recruiting patients were blinded to clinicians' allocation
Baseline (provider) characteristics similar	Low risk	"Both patient and clinician groups had similar sociodemographic characteristics"
Baseline (patient) characteristics similar	Low risk	See above
Baseline outcome measurement similar	High risk	Table 2 (the PAEI score was higher at baseline in the control group than in the intervention group ‐ 7.5 vs 6.3)
Blinding of outcome assessment (detection bias) All outcomes	High risk	"Neither clinicians nor research staff was blinded to the assignment of clinicians"
Adequately protected against contamination	High risk	All clinicians worked at 2 health centres ‐ so some of them must have worked together.
Incomplete outcome data (attrition bias) All outcomes	High risk	"Of the 13 clinicians randomized, 2 clinicians relocated and one retired, leaving 10 for analysis" Study authors stated that control group data were inadequate
Selective reporting (reporting bias)	Low risk	No reason to suspect
Other bias	Low risk	None identified
Risk of bias overall	High risk	High risk of bias overall

*Study characteristics*
Methods	Design: cluster RCT; randomised by institution
Participants	Country: USA Setting: community health centres Profession: staff Number of health professionals: 163 Number of patients: not reported Type of targeted behaviour: general management of a problem Proportion of eligible providers who participated: 50%
Interventions	• CME: resources (including educational materials) + training (consisted of training all health centre staff in a curriculum on improving asthma care in safety net practices, which was developed by using the NAEPP asthma care guidelines, and consisted of 2 duplicate sessions repeated in half‐day increments) + tools or templates for practice‐level systems change (examples of standing orders for peak flow measurement, as well as a template for an asthma flow sheet, were provided) • Control: group sites received copies of the national asthma guidelines and 1 asthma resource kit, with information provided on how they could obtain more at a discounted price Comparison 1: 1 vs 2
Outcomes	Providers: percentage of practitioner compliance with guideline indicators Patients: none
Notes	Number of behaviour change techniques: 3 Additional material to take home: yes Duration of educational meetings: 4 hours Intervention fidelity: proportion of attendance: 81% Use of theory: adult learning theory Interactive vs didactic educational meetings: interactive Complexity of targeted behaviour: low Importance of outcomes: low Time to follow‐up: not reported Decrease or increase in behaviour: increase Type of teacher: researcher Dates when study was conducted: not reported Funding: no information Declaration of interest: none
*Risk of bias*
Bias	Authors' judgement	Support for judgement
Random sequence generation (selection bias)	Unclear risk	"group randomized controlled trial"
Allocation concealment (selection bias)	Unclear risk	No information on allocation concealment
Recruiters blinded or recruitment taking place before randomisation	Unclear risk	Unclear whether recruiters were blind, or if recruitment of participants took place before randomisation
Baseline (provider) characteristics similar	Unclear risk	Provider baseline characteristics not reported
Baseline (patient) characteristics similar	High risk	Patient baseline characteristics not balanced on poverty level (69% vs 56% below), insurance status (34% vs 49% uninsured), and ethnicity (29.2% vs 68% African American or Hispanic) (see Table 1)
Baseline outcome measurement similar	High risk	Percentage of practitioner compliance with indicators not similar (fig 1 and fig 2)
Blinding of outcome assessment (detection bias) All outcomes	Low risk	"To protect confidentiality and to ensure provider buy‐in, data were collected in a manner that did not allow identification of the individual provider or patient"
Adequately protected against contamination	Low risk	Community health centres were randomised
Incomplete outcome data (attrition bias) All outcomes	High risk	"we had substantial dropout of centers randomized to either group ‐ 7/16 (44%) intervention and 9/16 (56%) control clusters that were enrolled had baseline and follow‐up data" "In our study, only one of the eight intervention sites was able to track asthma patients individually or in aggregate when they generated emergency department visits or hospital admissions"
Selective reporting (reporting bias)	Low risk	No reason to suspect
Other bias	Low risk	No evidence of other bias
Risk of bias overall	High risk	High risk of bias overall

*Study characteristics*
Methods	Design: RCT; randomised by provider
Participants	Country: Belgium Setting: hospital Profession: oncology nurses Number of health professionals: 116 from 33 hospitals Number of patients: 116 Type of targeted behaviour: communication skills Proportion of eligible providers who participated: 38%
Interventions	• CME: 3 weeks of training: 15 days, in total 105 hours (1 week each month in 3 months. 30 hours theoretical information, 75 hours role‐play and experiential exchanges. Topics: basic communication, psychosocial dimensions associated with cancer and its treatment, coping with patients' distress, how to discuss death and euthanasia. Use of 1 trainer, an experienced psychologist • Control: none (waiting list) Comparison 1: 1 vs 2
Outcomes	Providers: global measure of patient satisfaction with the interview Patients: none
Notes	Number of behaviour change techniques: 2 Additional material to take home: no Duration of educational meetings: 105 hours Intervention fidelity: proportion of attendance: 99% Use of theory: no Interactive vs didactic educational meetings: interactive Complexity of targeted behaviour: high Importance of outcomes: low Time to follow‐up: 6 months Decrease or increase in behaviour: increase Type of teacher: researcher Dates when study was conducted: no information Funding: grants of the ‘Fondation Kisane’, the ‘Commission Communautaire Francaise de la Region de Bruxelles‐Capitale de Belgique’, and the ‘Ministere de la Region Wallonne de Belgique’ Declaration of interest: no information
*Risk of bias*
Bias	Authors' judgement	Support for judgement
Random sequence generation (selection bias)	Unclear risk	"Every time 20 nurses were enrolled, the nurses were randomly allocated to a training group (TG) or to a control group (CG)"
Allocation concealment (selection bias)	Unclear risk	No information on allocation concealment
Recruiters blinded or recruitment taking place before randomisation	Low risk	Not applicable; not cluster randomised
Baseline (provider) characteristics similar	Low risk	"Baseline sociodemographic characteristics (age, sex, marital status, education) as well as socioprofessional characteristics (type of service, professional status, experience with cancer patients) were similar for both randomised groups (see Table 1)"
Baseline (patient) characteristics similar	Low risk	"The sociodemographic characteristics of cancer patients (age, sex, educational level, and setting of the interview) who participated at the different assessment points (at T1, N = 114; at T2, N = 111; at T3, N = 110) in the recorded interview with a HCP were also similar in the TG and the CG"
Baseline outcome measurement similar	Low risk	Baseline outcomes similar between groups (see Tables 2 to 5)
Blinding of outcome assessment (detection bias) All outcomes	Low risk	Low for patient questionnaires (EORTC QLQ‐C30; Satisfaction with the Interview Assessment Questionnaire): patients were most likely unaware of study allocation
Adequately protected against contamination	Unclear risk	116 nurses from 33 institutions were randomised and trained; it is possible that communication between intervention and control nurses could have occurred
Incomplete outcome data (attrition bias) All outcomes	Low risk	One nurse dropped out
Selective reporting (reporting bias)	Low risk	No reason to suspect reporting bias; all relevant outcomes in the methods section were reported in the results section
Other bias	Low risk	No evidence of other bias
Risk of bias overall	Unclear risk	Unclear risk of bias overall

*Study characteristics*
Methods	Design: provider RCT; randomised by provider
Participants	Country: Canada Setting: primary care Profession: pharmacists Number of health professionals: 59 Number of patients: N/A (standardised clients) Type of targeted behaviour: preventive care Proportion of eligible providers who participated: 40%
Interventions	• CME: asthma education programme: 1‐day workshop and 2 follow‐up telephone calls 2 and 4 weeks after the workshop • Control: delayed asthma education programme (received the intervention after outcomes were measured) Comparison 1: 1 vs 2
Outcomes	Providers: number of plans that a pharmacist facilitated during unannounced visits from 3 simulated clients Patients: none
Notes	Number of behaviour change techniques: 5 Additional material to take home: yes Duration of educational meetings: 8 hours Intervention fidelity: proportion of attendance: 86% Use of theory: not identifiable Interactive vs didactic educational meetings: interactive Complexity of targeted behaviour: low Importance of outcomes: low Time to follow‐up: 3 to 5 weeks Decrease or increase in behaviour: increase Type of teacher: researcher Dates when study was conducted: not reported Funding: this study was supported by an unrestricted educational grant from Merck Frosst Canada Inc., and in‐kind contribution from Agro Health Associates Inc., and the Centre for Evaluation of Medicines Declaration of interest: declared no conflict of interest
*Risk of bias*
Bias	Authors' judgement	Support for judgement
Random sequence generation (selection bias)	Low risk	"Pharmacists were randomized in blocks of six from a random numbers table"
Allocation concealment (selection bias)	Unclear risk	No information on allocation concealment
Recruiters blinded or recruitment taking place before randomisation	Low risk	Not applicable; not cluster randomised
Baseline (provider) characteristics similar	High risk	"...a notable difference between groups was seen in the number of prescriptions dispensed per month and the number of new prescriptions dispensed per month, and the number of telephone calls to a drug information center, which were all higher in the intervention group"
Baseline (patient) characteristics similar	Low risk	Unannounced simulated patients
Baseline outcome measurement similar	Unclear risk	No information on outcome measurement at baseline
Blinding of outcome assessment (detection bias) All outcomes	Low risk	"simulated patients (SPs) performing outcomes evaluation were blinded to assignment of the pharmacists to intervention or control groups"
Adequately protected against contamination	Unclear risk	Randomised by pharmacist; unclear whether they worked for the same pharmacy, in which case it is possible that communication between intervention and control professionals could have occurred
Incomplete outcome data (attrition bias) All outcomes	Low risk	4/33 (12%) and 1/30 (3%) dropped out from the intervention and control groups, respectively, with reasons provided
Selective reporting (reporting bias)	Low risk	No reason to suspect reporting bias; all relevant outcomes in the methods section were reported in the results section
Other bias	Low risk	No evidence of other bias
Risk of bias overall	High risk	High risk of bias overall

*Study characteristics*
Methods	Design: cluster RCT; randomised by health centre
Participants	Country: Egypt Setting: primary care Profession: primary healthcare physicians Number of health professionals: 40 practices Number of patients: 480 mother ‐ child Type of targeted behaviour: communication behaviours Proportion of eligible providers who participated: no information
Interventions	• CME: 3 days training programme + 'Child Card' with feeding instructions + training materials • Control: no intervention Comparison 1: 1 vs 2
Outcomes	Providers: % of mothers recalling nutritional advice Patients: gains in weight and length.
Notes	No baseline Number of behaviour change techniques: 3 Additional material to take home: yes ('Child Card') Duration of educational meetings: 24 hours Intervention fidelity: proportion of attendance: 70% Use of theory: no Interactive vs didactic educational meetings: interactive Complexity of targeted behaviour: low Importance of outcomes: low Time to follow‐up: 6 months Decrease or increase in behaviour: Type of teacher: specially trained: "five master trainers" Dates when study was conducted: no information Funding: Research Grant Agreement (03013 HNI) between WHO/HQ/CAH and the Faculty of Medicine, Suez Canal University, Egypt. Declaration of interest: "there are no conflicts of interest"
*Risk of bias*
Bias	Authors' judgement	Support for judgement
Random sequence generation (selection bias)	Low risk	"One center from each pair was selected randomly for the intervention group by flipping a coin and the other center was assigned to the control group"
Allocation concealment (selection bias)	Unclear risk	See above; no further information
Recruiters blinded or recruitment taking place before randomisation	Unclear risk	No information
Baseline (provider) characteristics similar	Low risk	"corrected for baseline"
Baseline (patient) characteristics similar	Low risk	"Recruited children and families of the intervention and control groups were comparable in their baseline characteristics (Table 1)"
Baseline outcome measurement similar	Unclear risk	No baseline measurements were done
Blinding of outcome assessment (detection bias) All outcomes	Low risk	"All observers, who were trained high school female graduates from the Ismailia governorate were blinded to the intervention or control status of the participants as well as the study objectives" ... "blind evaluation of the outcomes..."
Adequately protected against contamination	Low risk	Randomised by practice
Incomplete outcome data (attrition bias) All outcomes	Unclear risk	No information on practices (clusters) or providers. Less than 5% loss‐to‐follow‐up for patients ("...we did not use the ‘intention‐to‐treat’ principle in the data analysis")
Selective reporting (reporting bias)	Low risk	No reason to suspect
Other bias	Low risk	None identified
Risk of bias overall	Unclear risk	Unclear risk of bias overall

*Study characteristics*
Methods	Design: cluster RCT; randomised by community
Participants	Country: Iran Setting: primary care Profession: physicians Number of health professionals: 159 Number of patients: not reported Type of targeted behaviour: prescribing Proportion of eligible providers who participated: 61%
Interventions	• CME: 16 hours interactive EM ('outcome‐based education') • Control: 11 hours traditional CME programme Comparison 3: 1 vs 2
Outcomes	Providers: percentage of rational prescriptions Patients: none
Notes	Number of behaviour change techniques: 2 Additional material to take home: yes Duration of educational meetings: 16 hours Intervention fidelity: proportion of attendance: 78% Use of theory: adult learning theory Interactive vs didactic educational meetings: interactive Complexity of targeted behaviour: low Importance of outcomes: low Time to follow‐up: 3 months Decrease or increase in behaviour: decrease Type of teacher: professional colleague from same or other setting Dates when study was conducted: 2005 to 2006 Funding: not reported Declaration of interest: "disclosures: the authors report none"
*Risk of bias*
Bias	Authors' judgement	Support for judgement
Random sequence generation (selection bias)	Unclear risk	"allocated randomly"
Allocation concealment (selection bias)	Unclear risk	No information on allocation concealment
Recruiters blinded or recruitment taking place before randomisation	Unclear risk	Unclear whether recruiters were blind, or if recruitment of participants took place before randomisation
Baseline (provider) characteristics similar	Low risk	"Three cities from the north of the province were matched with three cities from the south based on a ranking of development factors such as economic status, health services, and education" GPs were similar on age, gender, and work experience
Baseline (patient) characteristics similar	Unclear risk	Baseline characteristics for patients were not reported
Baseline outcome measurement similar	Low risk	Prescribing patterns were similar between groups before the intervention (see Tables 4 and 5)
Blinding of outcome assessment (detection bias) All outcomes	Unclear risk	No information on blinding of outcome assessors
Adequately protected against contamination	Low risk	Randomised by city
Incomplete outcome data (attrition bias) All outcomes	Low risk	No cities dropped out; all but 1 enrolled GPs were included in analyses
Selective reporting (reporting bias)	Low risk	No reason to suspect reporting bias; all relevant outcomes in the methods section were reported in the results section
Other bias	Low risk	No evidence of other bias
Risk of bias overall	Unclear risk	Unclear risk of bias overall

*Study characteristics*
Methods	Design: cluster randomised controlled trial Follow‐up:: Providers: DONE Patients: NOT CLEAR Blinded assessment: DONE Baseline: NOT DONE Reliable outcomes: DONE Protection against contamination: NOT CLEAR Overall quality: MODERATE
Participants	Country: USA Setting: community‐based care Profession: nurses in a large urban home healthcare agency Number of health professionals: 205 Number of patients: 371 Medicare congestive heart failure patients Type of targeted behaviour: general management of a problem (care for patients with heart failure) Proportion of eligible providers who participated: not clear
Interventions	• CME: interactive practitioner training with role‐play and audiotaping • Control: no intervention Comparison 1: 1 vs 2
Outcomes	Professional practice: mean number of skilled nursing visits delivered within 90 days Patients: none
Notes	No baseline data Number of behaviour change techniques: 4 Additional material to take home: yes Duration of educational meetings: 3 hours Intervention fidelity: proportion of attendance: no information Use of theory: none Interactive vs didactic educational meetings: interactive Complexity of targeted behaviour: high Importance of outcomes: high Time to follow‐up: 3 months Decrease or increase in behaviour: decrease Type of teacher: researcher
*Risk of bias*
Bias	Authors' judgement	Support for judgement
Random sequence generation (selection bias)	Unclear risk	Method for random sequence generation not specified
Allocation concealment (selection bias)	Unclear risk	No information on allocation concealment
Recruiters blinded or recruitment taking place before randomisation	Unclear risk	It was not reported whether recruitment took place before randomisation, or if recruitment was blind (see page 5)
Baseline (provider) characteristics similar	Unclear risk	Text mentions characteristics (page 7), but data are not presented
Baseline (patient) characteristics similar	Low risk	Groups were similar
Baseline outcome measurement similar	Low risk	Groups were similar
Blinding of outcome assessment (detection bias) All outcomes	Unclear risk	No information on blinding of outcome assessors
Adequately protected against contamination	Unclear risk	Providers were allocated within an agency and communication between groups was possible
Incomplete outcome data (attrition bias) All outcomes	Unclear risk	Not reported
Selective reporting (reporting bias)	Low risk	No reason to suspect reporting bias; all relevant outcomes in the methods section were reported in the results section
Other bias	Low risk	No evidence of other bias
Risk of bias overall	High risk	High risk of bias overall

*Study characteristics*
Methods	Design: provider RCT
Participants	Country: Norway Setting: community‐based care Profession: public health physicians Number of health professionals: 148 Number of patients: Type of targeted behaviour: general management of a problem (working evidence‐based in a public health practice) Proportion of eligible providers who participated: 45%
Interventions	• CME: 1‐ to 5‐day workshop: (1 day: 10 physicians; 3 days: 21 physicians; 5 days: 18 physicians) + discussion list + help desk and information service + access to 5 databases + 3 newsletters • Control: no intervention Comparison 1: 1 vs 2
Outcomes	Professional practice: % of physicians having used research explicitly to some degree (used/not used) Patients: not applicable
Notes	Number of behaviour change techniques: 5 Additional material to take home: yes Duration of educational meetings: 24 hours Intervention fidelity: proportion of attendance: 68% Use of theory: Rogers' innovation diffusion theory Interactive vs didactic educational meetings: interactive Complexity of targeted behaviour: high Importance of outcomes: low Time to follow‐up: 18 months Decrease or increase in behaviour: increase Type of teacher: researcher Dates when study was conducted: "from April 1999 until the end of January 2001" Funding: Research Council of Norway Declaration of interest: "none declared"
*Risk of bias*
Bias	Authors' judgement	Support for judgement
Random sequence generation (selection bias)	Low risk	"Enrolled physicians were subsequently randomised to one of two groups by an independent researcher using computer software"
Allocation concealment (selection bias)	Low risk	... by an "independent researcher"
Recruiters blinded or recruitment taking place before randomisation	Low risk	Physicians recruited before randomisation
Baseline (provider) characteristics similar	Unclear risk	In the intervention group, 42% had attended session(s) in critical appraisal vs 30% in the control group, at baseline
Baseline (patient) characteristics similar	Low risk	Not applicable
Baseline outcome measurement similar	Low risk	Study authors adjusted "each respondent's follow‐up score with his or her baseline score. We expanded the model to also include baseline characteristics of possible prognostic strength"
Blinding of outcome assessment (detection bias) All outcomes	Low risk	"The registrar of the questionnaire data was blinded to group allocation. The researchers who scored the other study outcomes were blinded to the allocation of participants and whether the results were pre‐ or post‐tests"
Adequately protected against contamination	Low risk	1 physician in 1 municipality: "...physicians working in municipalities in Norway with more than 3000 inhabitants"
Incomplete outcome data (attrition bias) All outcomes	Unclear risk	"the response rate for reports which was 23% for the experiment group and 33% for the control group..." Assessed by the results of other outcomes; a higher response rate most likely would not have changed the conclusion
Selective reporting (reporting bias)	Low risk	No reason to suspect
Other bias	Low risk	None identified
Risk of bias overall	Unclear risk	Unclear risk of bias overall

*Study characteristics*
Methods	Cluster RCT; randomised by district
Participants	Country: Peru Setting: pharmacies, 14 districts of low socioeconomic status in Lima (pharmacies and physicians) Profession: pharmacists + physicians Number of health professionals: ca 200 (assessed) Number of patients: standardised simulated patients Type of targeted behaviour: general management of a problem (recognition, management, and prevention of STDs) Proportion of eligible providers who participated: 79%
Interventions	• CME: 1.5 hours × 3 = 4.5 hours luncheon training seminars + physicians in each district invited to attend a 6‐hour workshop on management of STD syndromes + referral network + monthly follow‐up visits for 6 months to all certified pharmacies and referral physicians and health centres within the district • Seminar on diarrhoea Comparison 1: 1 vs 2
Outcomes	Professional practice: recognition of STD symptoms, adequate management of the syndrome, recommendations for use of condoms, and recommendations for treatment of partners Patients: none (no data in our format)
Notes	Number of behaviour change techniques: 2 Additional material to take home: yes Duration of educational meetings: 4,5 hours Intervention fidelity: proportion of attendance: 84% Use of theory: no Interactive vs didactic educational meetings: interactive Complexity of targeted behaviour: low Importance of outcomes: high Time to follow‐up: 6 months Decrease or increase in behaviour: increase Type of teacher: researcher Dates when study was conducted: not clear, but study was published in 2003 and follow‐up after intervention was provided at 6 months Funding: "World AIDS Foundation Grant (WAF 155(97‐060)); NIH Fogarty International Center AIDS Research and Training Program (T22TW00001); National Institute of Mental Health Supplement D43TW00007; the University of Washington Center for AIDS Research (AI27757); and STD Cooperative Research Center (AI31448)" Declaration of interest: "none declared"
*Risk of bias*
Bias	Authors' judgement	Support for judgement
Random sequence generation (selection bias)	Low risk	"Within each pair, we flipped a coin to select one district to receive the STD intervention and one to serve as a control"
Allocation concealment (selection bias)	Unclear risk	Not clear
Recruiters blinded or recruitment taking place before randomisation	Unclear risk	No information, but seems as though districts were randomised, and then the pharmacies were recruited
Baseline (provider) characteristics similar	Unclear risk	No information
Baseline (patient) characteristics similar	Unclear risk	No information
Baseline outcome measurement similar	Unclear risk	"We did not do pre‐intervention assessments because our previous study of a random sample of 360 pharmacies throughout Lima found consistently low rates of recognition and discussions of the etiology of STD syndromes and very infrequent recommendations for condom use or for treatment of partners (15); these were similar to the rates found in the control districts in this study, with frequent offerings of ineffective antimicrobial drugs"
Blinding of outcome assessment (detection bias) All outcomes	Low risk	"We took several measures to minimize possible bias of the evaluators. The standardized simulated patients were not informed about the nature or design of the study. Specifically, they were not told that some districts did not receive the intervention, rather they were told we had trained pharmacy workers in all of Lima"
Adequately protected against contamination	Low risk	Randomised by district
Incomplete outcome data (attrition bias) All outcomes	Low risk	Lost 40% of clusters: "of 1118 pharmacies found in seven districts randomized to the intervention (75–301 per district), 884 (79%) agreed to participate; 750 (84%) of these completed training of all workers and were certified (57–76% of all pharmacies in the seven districts) [...] During follow‐up, 90 certified pharmacies closed; owners of these pharmacies opened 14 new pharmacies. At study end, 671 (89.4%) of the certified pharmacies still actively participated in the programme. New staff hired during the follow‐up period usually reported receiving training in STDs from co‐workers"
Selective reporting (reporting bias)	Low risk	No reason to suspect
Other bias	Low risk	None identified
Risk of bias overall	Unclear risk	Unclear risk of bias overall