Continuing education meetings and workshops: effects on professional practice and health care outcomes

Abstract

Background

Educational meetings are widely used for continuing medical education. Previous reviews found that interactive workshops resulted in moderately large improvements in professional practice, whereas didactic sessions did not.

Objectives

To assess the effects of educational meetings on professional practice and healthcare outcomes.

Search methods

We updated previous searches by searching the Cochrane Effective Practice and Organisation of Care Group Trials Register and pending file, from 1999 to March 2006.

Selection criteria

Randomised controlled trials of educational meetings that reported an objective measure of professional practice or healthcare outcomes.

Data collection and analysis

Two authors independently extracted data and assessed study quality. Studies with a low or moderate risk of bias and that reported baseline data were included in the primary analysis. They were weighted according to the number of health professionals participating. For each comparison, we calculated the risk difference (RD) for dichotomous outcomes, adjusted for baseline compliance; and for continuous outcomes the percentage change relative to the control group average after the intervention, adjusted for baseline performance. Professional and patient outcomes were analysed separately. We considered 10 factors to explain heterogeneity of effect estimates using weighted meta‐regression supplemented by visual analysis of bubble and box plots.

Main results

In updating the review, 49 new studies were identified for inclusion. A total of 81 trials involving more than 11,000 health professionals are now included in the review. Based on 30 trials (36 comparisons), the median adjusted RD in compliance with desired practice was 6% (interquartile range 1.8 to 15.9) when any intervention in which educational meetings were a component was compared to no intervention. Educational meetings alone had similar effects (median adjusted RD 6%, interquartile range 2.9 to 15.3; based on 21 comparisons in 19 trials). For continuous outcomes the median adjusted percentage change relative to control was 10% (interquartile range 8 to 32%; 5 trials). For patient outcomes the median adjusted RD in achievement of treatment goals was 3.0 (interquartile range 0.1 to 4.0; 5 trials). Based on univariate meta‐regression analyses of the 36 comparisons with dichotomous outcomes for professional practice, higher attendance at the educational meetings was associated with larger adjusted RDs (P < 0.01); mixed interactive and didactic education meetings (median adjusted RD 13.6) were more effective than either didactic meetings (RD 6.9) or interactive meetings (RD 3.0). Educational meetings did not appear to be effective for complex behaviours (adjusted RD ‐0.3) compared to less complex behaviours; they appeared to be less effective for less serious outcomes (RD 2.9) than for more serious outcomes.

Authors' conclusions

Educational meetings alone or combined with other interventions, can improve professional practice and healthcare outcomes for the patients. The effect is most likely to be small and similar to other types of continuing medical education, such as audit and feedback, and educational outreach visits. Strategies to increase attendance at educational meetings, using mixed interactive and didactic formats, and focusing on outcomes that are likely to be perceived as serious may increase the effectiveness of educational meetings. Educational meetings alone are not likely to be effective for changing complex behaviours.

Plain language summary

Continuing education meetings and workshops for health professionals

Educational meetings are commonly used for continuing medical education with the aim of improving professional practice and, thereby, patient outcomes. Educational meetings include courses, conferences, lectures, workshops, seminars, and symposia. 
 
 Eighty‐one trials that evaluated the effects of educational meetings were included in this review. Based on these studies, we concluded that educational meetings alone or combined with other interventions can improve professional practice and the achievement of treatment goals by patients. The effect on professional practice tended to be small but varied between studies, and the effect on patient outcomes was generally less. It is not possible to explain the observed differences in effect with confidence but it appeared that higher attendance at the meetings was associated with greater effects, that mixed interactive and didactic education was more effective than either alone, and that the effects were less for more complex behaviours and less serious outcomes.

Summary of findings

for the main comparison.

Educational meetings with or without other interventions compared to no intervention
Patient or population: Health care professionals Settings: Primary and secondary care Intervention: Educational meetings with or without other interventions* Comparison: No intervention
Outcomes	Adjusted absolute improvement (risk difference)† Median (Interquartile range)		Number of studies	Quality of the evidence (GRADE)	Comments
Compliance with desired practice	Median 6% (1.8 to 15.9)		30	+++O Moderate‡	The effect appears to be larger with higher attendance at the educational meetings and with mixed interactive and didactic educational meetings. Educational meetings did not appear to be effective for complex behaviours and they appeared to be less effective for less serious outcomes
Patient outcomes	Median 3.0% (0.1% to 4.0%)		5	+++O Moderate‡
* The effect of educational meetings alone on professional practice was the same as for multifaceted interventions that included educational meetings. †The post intervention risk differences are adjusted for pre‐intervention differences between the comparison groups. ‡We have downgraded the evidence from high to moderate because of inconsistency in the results that could not be fully explained.

GRADE Working Group grades of evidence 
 High quality (++++): Further research is very unlikely to change our confidence in the estimate of effect. 
 Moderate quality (+++O): Further research is likely to have an important impact on our confidence in the estimate of effect and may change the estimate. 
 Low quality (++OO): Further research is very likely to have an important impact on our confidence in the estimate of effect and is likely to change the estimate. 
 Very low quality (+OOO): We are very uncertain about the estimate.

Background

Each year, billions of dollars are spent worldwide on continuing medical education activities (Brown 2002; Vaughn 2006). Continuing professional development is another related but somewhat more comprehensive concept that emphasises a more self‐directed approach to education (Peck 2000). In many countries, a demonstration of continuing medical education is mandated by professional or regulatory bodies or it is stimulated by incentives (Peck 2000), which contribute greatly to the increase in these activities.

An underlying assumption is that continuing medical education improves healthcare practice and, thereby, health outcomes for patients. Two overviews of reviews on continuing medical education in general concluded that continuing medical education can be effective (Bloom 2005; Umble 1996) but the effect varied. The first overview was based on 16 reviews conducted between 1984 and 1994 and the other on 26 reviews from the period 1984 to 2004.

There has long been an awareness that the effectiveness of continuing medical education can be measured at three levels: competence, performance, and patient health status (Lloyd 1979) and that the impact declines in that order (Beaudrey 1989). Early studies in this field focused on establishing a causal relationship between continuing medical education and one or all of those outcome levels but, as this relationship was perceived to have been established, the focus shifted (Umble 1996). From the late 1980s, questions of how and why some programs worked better than others were raised and investigators looked for potential explanatory factors. Their focus also shifted from measuring knowledge, attitudes, or skills to measuring physicians' performance or patients' health. Commonly reported findings from explanatory analyses were that interventions using an interactive educational format had greater effects than those using a didactic format, and that multifaceted interventions had greater effects than single interventions (Mansouri 2007; Marinopoulos 2007).

Educational meetings are one of the most common continuing medical education activities (Brown 2002; Lloyd 1979). Educational meetings include courses and workshops in various formats. The nature of educational meetings is highly variable in terms of content, the number of participants, the degree and type of interaction, length, frequency, and the targeted practices. Other common continuing medical education activities are audit and feedback (Jamtvedt 2006) and educational outreach (O'Brien 2007), both of which are frequently combined with educational meetings. Quality improvement activities, which are closely related to continuing education (Boonyasai 2007), also commonly use small interactive meetings to facilitate learning and improvements in practice.

Previous versions of this review (Davis 1999; O'Brien 2001) assessed the effects of educational meetings and examined factors that could explain variations in effectiveness. These concluded that interactive workshops can result in moderately large changes in professional practice, while didactic sessions alone are unlikely to change professional practice. Another review of a wide range of guideline implementation strategies (Grimshaw 2004) concluded that educational meetings, with or without educational material, resulted in small to modest improvements when compared to no intervention, which is similar to other strategies.

In this update, we examined the effects of continuing education meetings on professional practice and patient outcomes. We also investigated factors that might influence the effectiveness of educational meetings. We used methods that have been developed by the Cochrane Effective Practice and Organisation of Care (EPOC) Group (Grimshaw 2003) since the previous review (O'Brien 2001). These methods were used in other recent EPOC reviews (Doumit 2007; Jamtvedt 2006; O'Brien 2007). The provision of printed educational materials has been reported to have little or no effect, in two reviews (Freemantle 1997; Grimshaw 2001), but this finding has been questioned in a more recent review (Grimshaw 2004). Because printed materials are usually an integral part of educational meetings, we chose to consider printed educational materials as a component of educational meetings and not as an additional independent intervention. Few studies have tested educational meetings without any printed educational materials (Grimshaw 2004).

Objectives

This review addressed the following questions:

1. Are educational meetings and workshops, alone or in combination with other interventions, effective in improving professional practice or healthcare outcomes?

Comparisons to answer the first question:

• Any intervention in which educational meetings is a component compared to no intervention (Comparison 1).

The primary aim of this analysis was to explore heterogeneity, including potential differences between the effects of educational meetings alone and educational meetings as a component of multifaceted interventions. The main explanatory factors that we considered were the:

• • type of intervention (educational meetings alone, with or without educational material, or multifaceted interventions that included educational meetings);

  • contribution of educational meetings as a component of the intervention for multifaceted interventions;

  • intensity of the educational meetings;

  • attendance at the educational meetings;

  • setting of care (primary care versus hospital);

  • interactive versus didactic educational meetings;

  • complexity of the targeted behaviour;

  • seriousness of the outcomes;

  • baseline compliance;

  • risk of bias (low, moderate, or high).

• Educational meetings compared to no intervention (Comparison 2).

2. How does the effectiveness of education meetings compare with that of other interventions?

Comparisons to answer the second question:

• Educational meetings compared to other interventions (Comparison 3).

3. Can educational meetings be made more effective by modifying how they are done?

Comparisons to answer the third question:

• Any intervention in which educational meetings are a component compared to educational meetings alone (Comparison 4).

• Interactive educational meetings compared to didactic (lecture‐based) educational meetings (Comparison 5).

• Any other comparison of different types of educational meetings (Comparison 6).

Methods

Criteria for considering studies for this review

Types of studies

Randomised controlled trials (RCTs) were included. Studies using quasi‐randomisation or other methods were excluded.

Types of participants

We included studies involving qualified health professionals or health professionals in postgraduate training (for example resident physicians). Studies involving only undergraduate students were excluded.

Types of interventions

We included the following types of educational meetings: conferences, lectures, workshops, seminars, symposia, and courses.

Types of outcome measures

We included studies that reported objectively measured health professional practice behaviours or patient outcomes in a healthcare setting. Studies that measured knowledge or performance in a test situation only were excluded. Studies using patients' (or simulated patients') subjective ratings of health professionals' performance were included.

Search methods for identification of studies

The review was updated using the Cochrane Effective Practice and Organisation of Care Group (EPOC) Trials Register and pending file. We identified all potentially relevant articles in the Register (see EPOC, Specialised Register http://www.mrw.interscience.wiley.com/cochrane/clabout/articles/EPOC/frame.html). We screened studies (1999 to March 2006) that were coded as an RCT or clinical controlled trial (CCT) and with the EPOC‐controlled vocabulary term 'educational meeting'. The EPOC pending file (studies identified using the EPOC search strategy and awaiting assessment) was also searched for the same period, by the EPOC Trials Search Coordinator. We included studies from the previous version of this review and did not undertake any additional searches for studies before 1999. The search history for the previous review is presented in Appendix 1. The reference lists of related systematic reviews and all obtained articles were screened.

An updated search was done in EMBASE (Appendix 1), Scopus, and the EPOC Trials Register (2006 to December 2007). Potentially relevant studies are listed under Studies awaiting classification.

Data collection and analysis

Two review authors (AB and LF) independently screened the titles and abstracts identified from the search process and eliminated any obviously irrelevant studies. The remaining studies were retrieved in full text. Two review authors (LF and AB, AR, GJ, MAOB, FW, or DA) independently applied inclusion criteria. Differences in opinion were resolved by discussions and the involvement of a third author.

Studies included in the previous review were reassessed because of changes in the data extraction form and the methods used in this updated review.

Risk of bias

The risk of bias for all included studies was independently assessed by two authors (LF and AB, AR, GJ, MAOB, FW or DD) using seven criteria suggested by EPOC for assessing the risk of bias of RCTs (see EPOC Data collection checklist http://www.epoc.cochrane.org/Files/Website/Reviewer%20Resources/Data%20Collection%20Checklist%20‐%20EPOC%20‐%202007‐Feb‐27.doc): concealment of allocation, follow up of professionals, follow up of patients or episodes of care, blinded assessment of primary outcome(s), baseline measurement, reliable primary outcome measure(s), and protection against contamination. An overall rating (low, moderate, or high risk of bias) was assigned based on these criteria. As a rule of thumb, studies were assigned a rating of low risk of bias if the first three criteria were scored as done, and there were no important concerns related to the last three criteria; moderate if one or two criteria were scored as not clear or not done; and high if more than two criteria were scored as not clear or not done. For cluster randomised trials protection against contamination was rated as done or not. We also rated concealment of allocation as done if all clusters were randomised at one time and there was no reason to suspect that the allocation process had been influenced by the investigators or participants. We rated completeness of follow up as done if data for at least 80% of the clusters in a cluster randomised trial were collected. However, if many practitioners or their patients had been lost to follow up, we assigned a rating of not done. Any discrepancies in ratings were resolved by discussion and the involvement of a third author.

Data extraction

Two authors independently completed data extraction for all studies. A revised version of the EPOC data collection checklist (see EPOC Data collection checklist http://www.epoc.cochrane.org/Files/Website/Reviewer%20Resources/Data%20Collection%20Checklist%20‐%20EPOC%20‐%202007‐Feb‐27.doc) was used to collect information on study design, type of intervention, presence of controls, type of targeted behaviour, participants, setting, methods (unit of allocation, unit of analysis, methodological quality), outcomes, and results. In addition data, as noted below, were registered. For studies with data that could not be extracted or that lacked baseline information, and that were not older than six to eight years, we contacted the investigators. Discrepancies between authors were resolved through discussion.

Description of explanatory factors

Type of intervention

We categorised interventions as educational meetings alone (with or without educational material) or as multifaceted interventions that included educational meetings. We defined multifaceted interventions as including two or more interventions, such as educational meetings and reminders. In some instances it was difficult to decide whether an intervention was primarily educational outreach, audit and feedback, or continuous quality improvement. In such cases, we used the investigators’ objective, research question, or description of the focus of the study to categorise the intervention. We used the following EPOC definitions (http://www.epoc.cochrane.org/Files/Website/Reviewer%20Resources/Data%20Collection%20Checklist%20‐%20EPOC%20‐%202007‐Feb‐27.doc) of interventions that might be combined with educational meetings:

• Reminders: any intervention, manual or computerised, that prompts the healthcare provider to perform some action

• Educational outreach: a personal visit by a trained person to health professionals in their own settings

• Audit and feedback: any summary of clinical performance of health care over a specified period of time, given in a written, electronic, or verbal format

Contribution of educational meetings

For multifaceted interventions, two of us independently and subjectively categorised the contribution of educational meetings as a component of the intervention: as a major, moderate, or minor component.

Intensity

We categorised the overall intensity of the educational meetings based on the following characteristics (with the categories listed from 'more intensive' to 'less intensive' for each characteristic):

• number of participants (small, moderate, or large group);

• format (interactive versus didactic);

• source (representatives coming from the local organisation versus a 'professionals' standards review organisation' (internal versus external organisation) or the researchers);

• frequency of the educational intervention, categorised as frequent (> 10), moderate (five to 10), infrequent (two to four), and once only;

• total length of education, categorised as prolonged (five days or more), moderate (two to four days), brief (one day), and very brief (less than one day).

Overall intensity was assessed by combining the above characteristics, as:

• intensive (small group AND interactive format AND a supervisor or senior colleague or representative from the local organisation as the source AND (frequent OR prolonged education));

• moderately intensive (any other combination of characteristics than those described in intensive or non‐intensive groups, such as small or moderate AND interactive or both interactive + didactic AND local or external organisation AND moderate frequency or moderate length of meeting);

• non‐intensive (small or moderate or large group AND (didactic format OR a 'professionals' standards review organisation' or representative of an external organisation or the investigators) AND (infrequently or once only) OR (brief or very brief)).

Attendance

If reported by the authors, we recorded the proportion of study participants that attended the educational session(s); if not, we estimated attendance on the basis of information in the text. If this was not possible, attendance was recorded as unknown.

Setting of care

We recorded the setting of care as general (family) practice, community‐based, hospital (inpatient or outpatient), mixed or other.

Format

We categorised educational meetings as interactive, didactic, or mixed. We defined didactic sessions as those that were predominantly lectures or presentations but which may have included question and answer periods. Interactive workshops and seminars were defined as sessions that involved some type of interaction amongst participants in small (< 10 participants), moderate (10 to 19 participants), or large (> 19 participants) groups. The interaction could include role play, case discussion, or the opportunity to practise skills. Mixed sessions included both didactic and interactive components. When in doubt, we categorised educational meetings as mixed.

Complexity

The complexity of the targeted behaviour was independently and subjectively categorised by two of us as: high, moderate, or low. The categories depended upon the number of behaviours required, the extent to which complex judgements or skills were necessary, and whether other factors such as organisational change were required for the behaviour to be improved; they also depended on whether there was a need for change by the individual or professional only (one person), communication change, or change in systems. If an intervention was targeted at relatively simple behaviours but there were a number of different behaviours, for example compliance with multiple recommendations for prevention, the complexity was assessed as moderate.

Seriousness

The seriousness (importance) of the outcome was independently and subjectively categorised by two of us as: high, moderate, or low. Acute problems with serious consequences were considered as high. Primary prevention was considered moderate. Numbers of unspecified tests or prescriptions were considered as low.

Baseline compliance

Baseline compliance with the targeted behaviours was treated as a continuous variable, ranging from zero to 100%, based on the pre‐intervention level of compliance given as a mean for both or all groups before the intervention.

Risk of bias

We categorised studies as having a low, moderate, or high risk of bias, as described above.

Data analysis

We only included in the primary analyses studies with a low or moderate risk of bias and that reported baseline data. For the first main comparison, we carried out a sensitivity analysis by including studies with a high risk of bias and baseline data. All outcomes were expressed as 'compliance with desired practice'. Professional and patient outcomes were analysed separately. Studies were weighted according to the number of health professionals participating.

To avoid the effect of potentially important baseline differences in compliance between the intervention and control groups of trials, the analyses were based on adjusted estimates of effect, where we adjusted for baseline differences in compliance. For dichotomous outcomes we calculated the adjusted risk difference.

The adjusted risk difference (RD) is the difference in compliance between intervention and control group means after the intervention minus the difference between groups before the intervention. A positive risk difference indicates that compliance improved more in the educational intervention group than in the control group, for example an adjusted risk difference of 0.09 indicates an absolute improvement in care (improvement in compliance) of 9%.

For continuous outcomes we calculated the percentage change relative to the control mean after the intervention: adjusted difference between the post‐intervention experimental and control group means divided by the post‐intervention control group mean x 100.

Comparisons that allocate clusters but do not account for clustering in the analysis have potential unit of analysis errors resulting in artificially low P values and overly narrow confidence intervals. For such comparisons, we extracted the point estimate and not the P value or confidence interval. For studies with no unit of analysis error and with low or moderate risk of bias and reported baseline data, we recorded adjusted odds ratios (or other measure of effect) and the P values or confidence intervals reported by the authors. We compared these results with our analyses to assess the robustness of our analyses.

When several outcomes were reported in a trial, we only extracted results for the primary outcome. If there was more than one primary outcome, or if the primary outcome was not specified, we calculated effect sizes for each outcome and extracted the median value across the outcomes. In the results tables, we tabulated the median adjusted risk difference (RD) in compliance for the primary outcomes for studies that reported an odd number of primary outcomes. For studies that reported an even number of primary outcomes, we chose the higher of the two middlemost adjusted RDs in compliance for the primary outcomes. In trials that reported summary as well as individual measures of performance, we used the summary measures.

Heterogeneity was explored visually by preparing tables, bubble plots (where the size of the bubble corresponds to the number of healthcare professionals who participated) and box plots (displaying medians, interquartile ranges, and ranges) to explore the size of the observed effects in relation to each of these variables. We considered each potential explanatory factor one at a time by looking for patterns in the distribution of the RDs. We hypothesised that greater effects would be associated with:

• multifaceted interventions (versus educational meetings alone),

• more intensive education meetings,

• higher attendance at educational meetings,

• interactive (versus didactic) educational meetings

• less complexity of the targeted behaviour,

• more serious outcomes,

• lower baseline compliance, and

• high risk of bias (versus moderate).

The visual analyses were supplemented with univariate statistical analyses. We used weighted meta‐regression to examine how the size of the effect was related to each of the 10 potential explanatory factors listed above, weighted according to the number of healthcare professionals. These analyses were conducted using generalised linear modelling in SAS (Version 9.1. SAS Institute Inc, Cary, NC, USA). We conducted the main analysis for the first comparison using the adjusted RD as the measure of effect.

We planned to supplement these univariate analyses with a multivariate meta‐regression. In order to minimise the risk of spurious estimates of effect from the meta‐regression, due to a high number of independent variables compared to the number of studies in the analysis, we planned to perform the meta‐regression in a stepwise manner with three steps:

1. Each of the potential explanatory factors were analysed as the only independent variable in a meta‐regression in order to assess an unadjusted baseline effect, variables with a P value > 0.3 were excluded as explanatory variables in step 3;

2. We examined interactions between the following factors and the type of intervention ‐ the intensity of educational meetings and interactive versus didactic educational meetings, interactions with a P value > 0.3 were excluded from further analysis;

3. Explanatory variables from step 1 (P value ≤ 0.3) and interactions from step 2 were evaluated for potential combination into a final meta‐regression model.

Publication bias

We used a funnel plot to visually explore the risk of publication bias, using the number of health professionals as a proxy for the precision of the estimate and the adjusted RD as the treatment effect.

Results

Description of studies

The search of the EPOC Trials Register and pending files (1999 to March 2006) yielded 768 references. The update of the EMBASE search retrieved references to 2355 studies, while the new search in the EPOC Trials Register and pending files returned 246 references. We identified 77 references which are now awaiting further assessment (see Characteristics of studies awaiting classification). The table Characteristics of excluded studies lists 33 references, including 14 references that were excluded from the original review.

In this update, 49 new studies have been added to the 32 studies from the previous review, making a total of 81 included studies. Of the 49 new studies, we contacted 20 investigators for further information regarding baseline data or extraction of data. Although 14 of these replied, we were only able to include one of those 14 studies in our analysis.

Characteristics of the providers and settings 
 Thirty‐two trials were based in North America (28 in the USA, four in Canada); 34 in Europe (14 in United Kingdom; 10 in the Netherlands; three in Norway; two in France; and one each in Sweden, Denmark, Belgium, Spain, and Scotland); three in Australia; two in Indonesia and South‐Africa; and one each in Mali, Thailand, Peru, Mexico, Zambia, Sri Lanka, New Zealand, and Brazil. In most trials the health professionals were physicians. In two studies the providers were nurses (Mazzuca 1987; Simons 2001), in three studies they were pharmacists (Kimberlin 1993; Chalker 2005; de Almeida Neto 2000) or non‐physician prescribers (Santoso 1996), and 18 studies involved mixed providers. The setting in 43 of the studies was categorised as general practice, 16 as community‐based care, 17 as hospital‐based care and five as other types of settings.

Targeted behaviours 
 In 11 trials the behaviours were preventive care, including: identifying and managing problems in marital relationships (Simons 2001; Thompson RS 2000), smoking cessation (Kottke 1989; Strecher 1991; Ward 1996), breastfeeding promotion activities (Westphal 1995), exercise and health behaviours counselling (Kerse 1999; Wilson 1992), screening sigmoidoscopy (Perera 1983), nutrition counselling (Ockene 1996), and follow up of patients with coronary artery disease (Kiessling 2002).

Three studies focused on test ordering behaviour change: improved quality of cholesterol testing (Van der Weijden 1999), and decreasing the number of tests requested (Verstappen 2003; Verstappen 2004). Six studies targeted screening behaviours for: cancer (Boissel 1995; Dietrich 1992; Dolan1997), cancer and hypertension (Jennett 1988), arthritis (Mazzuca 1987), and presentation of screening tests (Smith 1995).

Thirteen trials targeted prescribing: reducing antibiotic use (Angunawela 1991; Welschen 2004) or both antibiotics and steroids (Chalker 2005), identification of drug misuse (de Almeida Neto 2000), improved prescription of non‐steroidal anti‐inflammatory drugs (Figueiras 2001), reducing the use of injections (Hadiyono 1996), prescription of ACE inhibitors (Kasje 2004), prescribing indicators for upper respiratory tract infection (Meyer 2001), appropriate use of drugs for acute diarrhoea (Santoso 1996) or for asthma (Veninga 1999), adequate informing behaviour regarding prescribed drugs (Kimberlin 1993; Maiman 1988), and prescribing for osteoporosis (Solomon 2004).

Forty‐one trials focused on the general management of a wide array of problems. Behaviour was focused on in 41 trials with indicators for management of: low‐back pain (Bekkering 2005; Engers 2005), urinary tract infection and sore throat (Flottorp 2002), sexually transmitted diseases (Garcia 2003), depression (Gask 2004; Gerrity 1999; King 2002; Thompson C 2000; Worrall 1999), schizophrenia (Gray 2004), obstetric practices (Gülmezoglu 2006), preterm delivery (Leviton 1999), tuberculosis (Lewin 2005), obesity (Moore 2003 b), asthma (Smeele 1999), cardiovascular disease (Ornstein 2004), diabetes (Parker 1995; Varroud‐Vial 2004; Woodcock 1999), acute myocardial infarction (White 1985), epilepsy (Davis 2004), angina (Heller 2001), infertility (Morrison 2001), neonatal care (Wirtschafter 1986), hypercholesterolaemia (Browner 1994), and congestive heart failure (Feldman 2004). Other studies allocated to this category were several studies targeted at improving communication skills: for physicians (Brown 1999; Clark 1998; Delvaux 2005; Fallowfield 2002; Gilroy 2004; Harmsen 2005; Levinson 1993; Roter 1995) or to a related field, improvement of dietary consultations (Moore 2003 a). Single studies sought to increase the use of active sick leave (Scheel 2002), brief therapy training (Pekarik 1994), and research evidence in public health (Forsetlund 2003); and to improve overall quality management (Bexell 1996), referral practice (Rowlands 2003), and guideline‐consistent behaviour (Schectman 2003). Heale 1988 compared the effect of interactive versus didactic education.

The other studies targeted: handling of frequent attenders at an out‐of‐hours service (Christensen 2004), improvement of skills in spirometry (Eaton 1999), the rate of back surgery (Goldberg 2001), positioning of stroke patients (Jones 1998), patients' trust building (Thom 1999) and promotion of the use of advance directives for end of life decisions (Sulmasy 1996).

Characteristics of the intervention

Thirty‐two trials tested multifaceted interventions. The most commonly used co‐interventions were: reminders (five studies), patient education materials (five studies), supportive services (five studies), feedback reports (10 studies), and educational outreach (five studies). In 12 of these studies educational meetings were rated as: the main component of the intervention, moderate in 13 studies, and as a minor component in seven. We categorised one study as having intensive educational meetings (Gilroy 2004), 25 as moderately intensive, and 54 as non‐intensive meetings. In addition, one study compared moderate and low‐intensity educational meetings (Browner 1994). Out of the 81 studies, 44 (54%) had an attendance of 80% or more. In 21 studies (26%) it was unclear how many had attended the meetings. 
 
 Twenty‐three studies examined the effectiveness of interactive educational meetings and 10 studies examined the effectiveness of didactic educational meetings, while 43 studies tested a mixed format. In three studies it was not clear how the intervention should be characterised. In addition, two studies had more than two arms. These compared interactive, didactic, and mixed formats (Heale 1988); or an interactive format compared to didactic (Santoso 1996).

In 14 (17%) studies it was stated explicitly that the intervention built on a known theory for behaviour change, learning theory, or diffusion of innovations theory.

Outcome measures

There was large variation in the number of outcome measures, as well as what was being measured; 62% of the studies used dichotomous outcomes measures, 32% used continuous measures, and four studies used both types of measures. Professional practice, patient outcomes, or both, were studied in: 58 (72%), 9 (11%), and 14 (17%) of the studies respectively. The time to follow up varied from 14 days to two years, with a median follow up of six months.

Risk of bias in included studies

We judged 17 studies to have a low risk of bias, 44 a moderate risk, and 20 a high risk. In 47 trials, we assessed that the allocation of participants to experimental and control groups was adequately concealed. For all but one of the remaining trials, adequacy of concealment could not be determined from the published reports. Outcomes were assessed blindly in 50 of the 81 studies, with for all but three of the remaining studies blinding was assessed as not clear. It was often difficult to assess the loss to follow up, of practices or professionals; for example sometimes the number of health personnel was not reported at all, only the number of clusters. We tried to balance these two considerations; that is if data for all clusters were reported we did not rate follow up as done if more than 20% of participants had been lost to follow up. We assessed 52 (64%) of the trials as having over 80% follow up of participants, 17 trials (21%) as not clear, and 12 (15%) as having less than 80% follow up of the units randomised.

Effects of interventions

See: Table 1

Comparison 1: any intervention in which educational meetings were a component compared to no intervention

There were 80 trials involving more than 11,000 health professionals in this comparison. However, 20 of the studies were judged as having a high risk of bias, 13 studies had no baseline data, and sufficient data could not be extracted from three studies. Of the 44 remaining studies, eight studies reported data for continuous outcomes and six studies reported patient outcomes only. Data for each trial in this comparison can be found at http://www.epoc.cochrane.org/en/newPage2.html.

Professional practice 
 Thirty trials with 36 comparisons reported dichotomous health professional outcomes, had a low or moderate risk of bias, and reported baseline data. The adjusted RDs in compliance with desired practice varied from ‐2.0% to 36.2%, with a median improvement of 6% (interquartile range 1.8% to 15.9%).

When we included the studies judged as having a high risk of bias but that had baseline data (five studies with five comparisons) in a sensitivity analysis it did not change the overall results. The adjusted RDs in compliance with desired practice varied from ‐2.0% to 36.2% with a median improvement of 6% (interquartile range 2.0% to 14.7%). 
 
 The 36 comparisons from the 30 trials were included in univariate meta‐regression analyses. There were six factors and two interactions from the univariate analyses that had P < 0.3:

• contribution of educational meetings (P = 0.06);

• attendance at the meetings (P = 0.01);

• interactive versus didactic meetings (P = 0.03);

• complexity of the targeted behaviour (P = 0.02);

• seriousness of the outcome (P = 0.02);

• risk of bias (P = 0.28);

• interaction between interactive versus didactic meetings and multifaceted interventions (P = 0.003);

• interaction between intensity of educational meeting and multifaceted interventions (P = 0.21).

Because of the large number of variables (eight) that were eligible for inclusion in the multivariate analysis relative to the number of included comparisons (36), we chose not to carry out the planned multivariate analysis. This was due to the high risk of spurious findings. For the same reason, the P values from the univariate analyses must be interpreted cautiously. Inspection of the bubble and box plots for the four most statistically significant explanatory factors (P < 0.03) suggested some inconsistent relationships.

Higher attendance at the educational meetings was associated with larger adjusted RDs, as hypothesised (P < 0.01).

Mixed interactive and didactic education meetings (median adjusted RD 13.6) were more effective than didactic meetings (RD 6.9), as hypothesised; but interactive meetings appeared to be less effective (RD 3.0) (Figure 1).

1.

Box plot of adjusted risk difference (RD) versus the type of education

For complexity of the targeted behaviour, the hypothesis was that the more complex the behaviour the smaller the effect would be. This was the case for high complexity (adjusted RD ‐0.3) versus moderate (RD 10.5) or low (RD 4.7) complexity behaviours, but not for moderate versus low complexity behaviours.

Similarly, we had hypothesised that the more serious the outcome, the greater the effect on the targeted behaviour. This was the case for a high level of seriousness of the targeted behaviour (adjusted RD 7.1) or moderate (RD 9.8) versus low (RD 2.9), but not for high versus a moderate level of seriousness.

The differences in effect estimates between studies with a multifaceted intervention and studies with educational meetings alone were not statistically significant (median adjusted RD 6.0 for both, P = 0.90); and for studies with different baseline compliance rates (P = 0.8).

Eighteen trials reported continuous outcomes, of which eight trials had baseline data and a low or moderate risk of bias. The adjusted relative percentage change varied from 0% to 53%. The median percentage change was 10% (interquartile range 9% to 24%).

Patient outcomes 
 There were 21 trials that reported patient outcomes in this comparison. Out of the 21 trials, 13 trials were of low or moderate risk of bias and had baseline values. For dichotomous outcomes, there were five trials. The adjusted RDs in the achievement of treatment goals varied from ‐0.9% to 4.6%, with a median improvement of 3.0% (interquartile range 0.1% to 4.0%).

For continuous outcomes there were eight studies (nine comparisons). The adjusted relative percentage change in the patient health indicator varied from ‐1% to 26%. The median percentage change was 4% (interquartile range 0% to 11%).

See Table 1.

Comparison 2: educational meetings alone compared to no intervention

There were 56 trials in this comparison, of which 41 trials had dichotomous outcomes and 15 had continuous health professional outcomes. Twenty‐four trials (26 comparisons) of educational meetings alone compared to no intervention were judged to have a low or moderate risk of bias and reported baseline data. Data for each trial in this comparison are available at www.epoc.uottawa.ca.

Professional practice 
 Of the 24 trials (26 comparisons), 19 trials (21 comparisons) had dichotomous data. The data from these trials were used to calculate the median and interquartile range. The median adjusted RDs varied from ‐2.0% to 29.3%, with a median of 6% (interquartile range 2.9% to 15.3%).

Five trials (five comparisons) out of the 24 studies reported continuous outcomes. The data from these trials were used to calculate the median and interquartile range. The median adjusted relative percentage changes ranged from 0% to 50%, with a median of 10% (interquartile range 8% to 32%).

Patient outcomes

Seventeen trials reported patient data but only nine studies were judged to be of moderate or low risk of bias and had baseline data. Out of these, there were three trials reporting dichotomous data and six trials comparing continuous data. The adjusted RDs for dichotomous patient outcomes varied from ‐0.9 to 4.0 with a median improvement of 3.0 (interquartile range ‐0.9 to 4.0). For continuous patient outcomes the adjusted relative percentage change varied from ‐1% to 26%, with a median of 8% (interquartile range 0% to 12.0%).

Comparison 3: educational meetings compared to other interventions

Two trials compared educational meetings to other interventions. Both trials had a moderate risk of bias and used dichotomous outcomes. The comparison interventions were: a facilitated implementation of an office system to improve services for early detection of cancer (Dietrich 1992), and an educational outreach visit intervention to increase prescribing of recommended non‐steroidal anti‐inflammatory drugs for patients with osteoarthritis and inflammation (Figueiras 2001). The adjusted RD for the comparison educational meetings versus office meetings was a ‐8.0% decrease in compliance for the educational intervention group. For the comparison of educational meetings versus educational outreach, the adjusted RD was ‐1.4% decrease in compliance for the educational intervention group.

No patient outcomes were reported in these trials.

Comparison 4: any intervention in which educational meetings were a component compared to educational meetings alone

There were seven trials in which a multifaceted intervention that included educational meetings was compared to educational meetings alone. Only one of these trials had a low or moderate risk of bias and reported baseline data (Dietrich 1992). This study compared one‐day small group discussions combined with an office system and facilitator with a one‐day small group discussions only. The aim was to improve services for early detection of cancer. There was a 12% adjusted relative percentage increase in patients receiving faecal occult blood testing. Another study had a low risk of bias but no baseline data (Browner 1994). A three‐hour seminar plus follow‐up seminars, patient education materials, and office visits to medical doctors and other staff, plus reminders, were compared to a three‐hour seminar. The authors reported that the proportion of patients being screened for hypecholesterolemia was the same for both groups (51%) and the proportion of patients being managed in compliance with guidelines was 34% in the intensive group and 33% in the education only group.

Comparison 5: interactive educational meetings compared to didactic (lecture based) educational meetings

Two trials compared interactive to didactic educational meetings. Only one of these had a low or moderate risk of bias and reported baseline data (Santoso 1996). The aim of this study was to improve appropriate drug use in acute diarrhoea, to prevent dehydration and death. Locally arranged interactive educational meetings were compared to didactic educational meetings. These were arranged for all prescribers in a health district. Although a somewhat larger improvement was reported for the group receiving interactive education it was not statistically significant (adjusted RD 1.4%).

Comparison 6: any other comparison of different types of educational meetings

One study which was judged as having a high risk of bias; it had no baseline data (Heale 1988).

Publication bias

We visually explored a funnel plot of the 36 studies included in the main comparison (Figure 2). The plot was asymmetrical with more smaller studies spreading far out to the right (a larger adjusted RD) and not to the left (a RD less than zero). While this may be an indication of publication bias, there are other possible explanatons for this asymmetry, including poorer methodological quality of the smaller studies, true heterogeneity (for example due to the smaller studies having higher attendance rates), and chance (Cochrane Handbook 2008). It is also plausible that educational meetings may occasionally have (true) large effects and that they rarely have large negative impacts on professional practice. Although, we cannot draw firm conclusions about the existence of publication bias, the asymmetry suggests that studies that include fewer than 100 healthcare professionals may sometimes overestimate the impact of educational meetings on professional practice.

2.

Discussion

The main finding of this updated review is consistent with earlier versions of the review. This is that educational meetings can result in small to moderate improvements in professional practice and, as would be expected (Umble 1996), smaller improvements in patient outcomes. However, the results of this update suggest that improvements are most likely to be small (median adjusted RD 6%) even with educational meetings that vary in their impact. The explanation of this, emerging from this update, differs from the previous version of the review. We have included more than twice as many studies as in the previous version and potential explanations for the observed variation in effects are more complex than emerged with the smaller number of studies.  
 
 A key finding of the previous review, as well as other reviews (for example Bloom 2005), was that interactive education was more effective than didactic education, which appeared to have little or no effect on professional practice. In this update, we found that mixed interactive and didactic education was most effective, whereas interactive education alone appeared to be least effective; the median adjusted RD for didactic education was 6.9%. A possible explanation for this apparent inconsistency is that the studies often provided minimal descriptions of the interventions, making it difficult to classify them. When in doubt, we categorised interventions as mixed. In future updates, and in practice, it may be more relevant to only use two categories. These are didactic educational meetings and educational meetings that are partially or largely interactive. In the context of this review this would suggest that while didactic education can change professional practice, educational meetings that are partially or largely interactive appear to be more effective.

We did not find a significant difference in the effects of multifaceted interventions and educational meetings alone. The median adjusted RD for both was 6%. This finding of similar effects for multifaceted and single interventions is consistent with the results of two other reviews (Grimshaw 2004; Jamtvedt 2006), although some reviews have concluded that multifaceted interventions are more effective (for example Grimshaw 2001; Mansouri 2007).

Among the other explanatory factors that we explored in this update, those that may help to explain variations in the impact of educational meetings on professional practice are the proportion of professionals in the target audience that attend the meetings, the complexity of the targeted behaviour, and the seriousness or importance of the targeted outcome. The intensity of the educational meetings, the setting, baseline compliance, and the risk of bias did not help to explain the observed variation in effects. These findings should be interpreted cautiously because they are based on indirect (between study) comparisons. There were a large number of potential explanatory factors (10) relative to the number of comparisons included in our primary analysis (36), and several of the explanatory factors were difficult to code. Nonetheless, these findings may provide some useful insights for those planning and evaluating educational meetings.

It is logical that people who do not attend educational meetings would not benefit from them and, therefore, the impact on professional practice would decrease as the proportion of people in the target audience that attend the meetings declines. Only 54% of the included studies had an attendance of 80% or more. Health professionals may select continuing educational activities in areas in which they are interested and, therefore, already performing well. For this reason, the impact on those who did not attend could potentially be greater than on those who did; had they attended (Sibley 1982). Thus, those planning educational meetings may want to consider strategies to increase attendance, particularly amongst those who might not choose to attend based on prior interests, in order to increase the potential impact on targeted practices.

Our findings also support the logic that educational meetings are unlikely to improve practice for highly complex behaviours (median adjusted RD ‐0.3). Similarly, they suggest that the impact of educational meetings may be smaller for outcomes that health professionals may perceive as not having serious consequences for patients (for example the total number of tests ordered) (median adjusted RD 2.9) compared to outcomes that they may perceive as having moderately or highly serious consequences for patients (median adjusted RD 9.8 and 7.1, respectively).

Although we did not find statistically significant differences in the effects of educational meetings on professional practice, there was a trend suggesting that more intensive interventions might have larger effects, as would be expected. We categorised most of the interventions as non‐intensive (27 of the 36 comparisons), none as intensive, and the rest as moderately intensive. Thus we cannot draw conclusions about ‘intensive’ interventions based on these data and it is possible that the approach we used to categorise the intensity of interventions was not adequate to detect relevant differences in intensity amongst the included interventions.

We found only two studies that tested different ways of modifying educational meetings (Dietrich 1992; Santoso 1996). Thus there is extremely limited information from direct comparisons of different types of educational meetings to supplement our indirect comparisons and inform decisions about how to modify educational meetings to make them more effective.

We found only two studies that compared the impact on professional practice of educational meetings and other interventions. These studies found that educational meetings alone were less effective than office systems, and slightly less effective than educational outreach (Dietrich 1992; Figueiras 2001), but no firm conclusions can be drawn about the effectiveness of educational meetings compared to other interventions based on these results.

Other systematic reviews of educational outreach (O'Brien 2007) and audit and feedback (Jamtvedt 2006) have used similar methods to our review. These reviews also omitted studies with high risk of bias from the analysis. The median adjusted RD for professional practice for educational outreach compared to no intervention was 5% (interquartile range 3.0% to 6.2%). The median adjusted RD for audit and feedback compared to no intervention was 4% (interquartile range ‐0.8% to 9%). These findings are similar to our findings for educational meetings (median adjusted RD 6%, interquartile range 2.9% to 15.3%) suggesting that, with all the limitations of indirect comparisons, the effectiveness of these three interventions may be similar.

Limitations

As with any systematic review, our review is limited by the data provided in the included studies. Of the 81 studies that met our inclusion criteria, we judged 20 studies to have a high risk of bias and did not, therefore, include them in our primary analyses. Of the remaining 61 studies, only 30 provided data that could be included in meta‐regression analyses exploring the heterogeneity in the effects of educational meetings on professional practice. Thus, despite a large number of relevant studies, these studies provide only a limited basis for informing decisions about when educational meetings are most likely to be effective or how best to design implementation strategies using educational meetings. Also, there may be several relevant studies among the many studies awaiting assessment.

Our findings are further limited by inadequate descriptions of the interventions in many of the included studies, as well as by our ability to characterise studies with respect to the many potential factors that could explain the heterogeneity in the results of the included studies. In particular, we found it difficult to characterise the contribution of educational meetings to multifaceted interventions, the intensity of educational meetings, the format (interactive versus didactic), the complexity of the targeted behaviours, and the seriousness of the targeted outcomes. The need for difficult judgements, and an inadequate basis for making many of these judgements, adds to the need to interpret the results of our analyses cautiously.

The scope of our review is both a strength and a limitation. Not restricting our scope to a specific clinical problem or area increased the number of studies that could be included and reduced the risks of spurious findings. However, it is not possible to draw firm conclusions based on this review regarding the effects of educational meetings for specific clinical problems, or how best to design educational meetings for specific clinical problems. Nonetheless, we would argue that our review provides a useful context in which to interpret the findings of the individual trials included in this review as well as other studies that address more specific questions about the effects of educational meetings.

 

Authors' conclusions

Implications for practice.

Educational meetings, alone or combined with other interventions, can improve professional practice and patient outcomes. The effect is most likely to be small and similar to other types of continuing medical education, such as audit and feedback and educational outreach visits. Strategies to increase attendance at educational meetings, using mixed interactive and didactic formats and focusing on outcomes that are likely to be perceived as serious, may increase the effectiveness of educational meetings. Educational meetings alone are not likely to be effective for changing complex behaviours.

Implications for research.

Future reports of trials of continuing medical education should include clear and detailed descriptions of the interventions, including the proportion of the target audience that attended, the size of groups at meetings, the length and number of sessions, the teaching techniques, and whether there was any skills practice. They should adhere to the CONSORT recommendations for reporting RCTs (Altman 2001), including the extensions for cluster randomised trials (Campbell 2004) and for pragmatic trials (Zwarenstein 2008).

Further comparisons of educational meetings alone that are targeted at individuals at one level of an organisation and no intervention are unlikely to further our understanding of when educational meetings are likely to be effective and how to improve their effectiveness. Direct comparisons of different types of education are needed, such as different group sizes and different numbers and lengths of follow up. In particular, evaluations of more intensive interventions, which may be both more effective and more costly, compared to less intensive interventions are needed. Evaluations of conceptual models or theories to tailor continuing medical education in order to maximise its effectiveness are also needed. These evaluations should use cluster randomised designs, whenever possible, together with process evaluations to further our understanding of why interventions do or do not work and the variations in their effects (Northstar).

For the next update of this review we will reconsider the categorisation of potential explanatory factors and re‐evaluate the use of a multivariate regression analysis.

What's new

Date	Event	Description
11 October 2012	Amended	Reference for Orstein 2004 fixed

History

Review first published: Issue 2, 2001

Date	Event	Description
12 February 2009	New search has been performed	All searches updated.
12 February 2009	New citation required and conclusions have changed	This is an update of previously published review. Forty‐nine new studies have been added to the 32 studies from the previous review, making a total of 81 included studies. The search was re‐run in December 2007: Seventy‐seven references are listed under 'Studies awaiting classification'.
19 June 2008	Amended	Converted to new review format.

Appendices

Appendix 1. Search strategies

Search strategy for previous review

The previous review (O'Brien 2001) searched the Cochrane Effective Practice and Organisation of Care (EPOC) Group Specialised Register, MEDLINE (1966 to January 1999) without language restrictions, and the Research and Development Resource Base in Continuing Medical Education (RDRB/CME) (Davis 1991). The reference lists of related systematic reviews and all articles obtained were reviewed. The terms for the MEDLINE search follow: education/; exp education,continuing/; exp education,graduate/; internship and residency/; exp inservice training/; preceptorship/; exp teaching/. The educational terms were combined with methodological terms.”

Davis D, Rox R. The Research and Development of Resource Base in CME. An Annotated Bibliography and Literature Searching Service. Annual Report. Hamilton: McMaster University, 1991.

Database: EMBASE <1980 to 2007 Week 49>

Search Strategy:

‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐

1     *medical education/ (27217)

2     *continuing education/ (4249)

3     *postgraduate education/ (2708)

4     ((education$ or train$) adj (program$ or intervention? or meeting? or session? or strateg$ or workshop?)).tw. (30225)

5     ((education$ or train$) adj (lecture? or symposi$ or course?)).tw. (1947)

6     or/1‐5 (61610)

7     (random$ or placebo$).tw. (402856)

8     ((single$ or double$ or triple$ or treble$) and (blind$ or mask$)).tw. (94419)

9     controlled clinical trial?.tw. (9545)

10     or/7‐9 (423112)

11     6 and 10 (5020)

12     Animals/ (18216)

13     Humans/ (5951720)

14     12 not (12 and 13) (14454)

15     11 not 14 (5020)

16     limit 15 to yr="2003 ‐ 2007" (2210)

Characteristics of studies

Characteristics of included studies [ordered by study ID]

Angunawela 1991.

Methods	Cluster RCT Follow up: providers: DONE patients: N/A Blinded assessment: DONE Baseline: NOT CLEAR Reliable outcomes: NOT CLEAR Protection against contamination: DONE Overall quality: HIGH
Participants	43 prescribers in 15 state health institutions and patients (18 766 episodes of care) Country: Sri Lanka Proportion of eligible providers who participated: 94% Outpatient departments; Academic/Teaching setting: MIXED Type of targeted behaviour: PRESCRIBING (antibiotics) Complexity of targeted behaviour: LOW
Interventions	1. CME: didactic seminar 3 hrs + printed material 2. Printed material 3. No intervention control
Outcomes	Professional practice: % patients receiving prescriptions for antibiotics Patient: none Seriousness of outcome: LOW
Notes
Risk of bias
Bias	Authors' judgement	Support for judgement
Allocation concealment (selection bias)	Low risk	A ‐ Adequate

Bekkering 2005.

Methods	Cluster RCT Follow up: providers: NOT CLEAR patients: NOT CLEAR Blinded assessment: DONE Baseline: NOT DONE Reliable outcomes: NOT DONE Protection against contamination: DONE Overall quality: MODERATE
Participants	113 physiotherapists in 68 practices Country: Netherlands Proportion of eligible providers who participated: 21% Community‐based care Academic/Teaching setting: NOT CLEAR Type of targeted behaviour: GENERAL MANAGEMENT OF A PROBLEM (low back pain) Complexity of targeted behaviour: LOW
Interventions	1. CME: 2.5 hrs x 2 didactic and interactive workshops targeted at barriers 2. No intervention control (guidelines by mail)
Outcomes	Professional practice: proportion of adherence to guidelines for four recommendations Patient: % of patients at sick leave during previous 6 weeks at 52 weeks Seriousness of outcome: LOW
Notes
Risk of bias
Bias	Authors' judgement	Support for judgement
Allocation concealment (selection bias)	Low risk	A ‐ Adequate

Bexell 1996.

Methods	Cluster RCT Follow up: providers: DONE patients: N/A Blinded assessment: DONE Baseline: DONE Reliable outcomes: NOT CLEAR Protection against contamination: DONE Overall quality: HIGH
Participants	Prescribers (clinical officers and medical officers) in 16 health centres Country: Zambia Proportion of eligible providers who participated: 84% Community‐based care Academic/Teaching setting: NOT CLEAR Type of targeted behaviour: GENERAL MANAGEMENT OF A PROBLEM (quality of patient management and rational drug use) Complexity of targeted behaviour: LOW
Interventions	1. CME: 3‐day interactive seminar x 2 2. No intervention control
Outcomes	Professional practice: overall proportion of patients adequately managed Patient: none Seriousness of outcome: LOW
Notes
Risk of bias
Bias	Authors' judgement	Support for judgement
Allocation concealment (selection bias)	Low risk	A ‐ Adequate

Boissel 1995.

Methods	Cluster RCT Follow up: providers: NOT CLEAR patients: N/A Blinded assessment: NOT CLEAR Baseline: NOT DONE Reliable outcomes: NOT CLEAR Protection against contamination: DONE Overall quality: LOW
Participants	385 general practitioners in 278 practices providing breast and cervical cancer screening Country: France Proportion of eligible providers who participated: NOT CLEAR Primary care Academic/Teaching setting: NOT CLEAR Type of targeted behaviour: SCREENING (for cancer) Complexity of targeted behaviour: LOW
Interventions	1. CME: one‐day seminar and educational material sent four times over one year 2. No intervention control
Outcomes	Professional practice: average number of prescriptions for mammography and smear tests Patient: none Seriousness of outcome: LOW
Notes	No baseline data
Risk of bias
Bias	Authors' judgement	Support for judgement
Allocation concealment (selection bias)	Unclear risk	B ‐ Unclear

Brown 1999.

Methods	Cluster RCT Follow up: providers: DONE patients: N/A Blinded assessment: NOT CLEAR Baseline: DONE Reliable outcomes: NOT DONE Protection against contamination: NOT CLEAR Overall quality: MODERATE
Participants	70 primary care physicians, surgeons, medical subspecialists, physician assistants and nurse practitioners Country: USA Proportion of eligible providers who participated: 7% Community‐based care Academic/Teaching setting: NOT CLEAR Type of targeted behaviour: COMMUNICATION BEHAVIOUR Complexity of targeted behaviour: LOW
Interventions	1. CME: didactic and interactive workshop, 8 hrs 2. No intervention control
Outcomes	Professional practice: average score on the Art of Medicine Survey Patient: none Seriousness of outcome: LOW
Notes
Risk of bias
Bias	Authors' judgement	Support for judgement
Allocation concealment (selection bias)	Unclear risk	B ‐ Unclear

Browner 1994.

Methods	Cluster RCT Follow up: providers: DONE patients: N/A Blinded assessment: DONE Baseline: NOT DONE Reliable outcomes: NOT CLEAR Protection against contamination: DONE Overall quality: HIGH
Participants	197 primary care physicians in 174 practices Country: USA Proportion of eligible providers who participated: 65% GPs/family practitioners Academic/Teaching setting: NON‐TEACHING setting Type of targeted behaviour: GENERAL MANAGEMENT OF A PROBLEM (screening for hypercholesterolemia) Complexity of targeted behaviour: MEDIUM
Interventions	1. Intensive CME: didactic and interactive seminar: 3 hrs + 2 hrs seminar + a third seminar a couple of months later + phone calls + 2 visits to MD and staff to explain educational material + laminated cards + chart reminders + post‐card reminders to patients 2. Standard CME: didactic and interactive seminar: 3 hrs 3. No intervention control
Outcomes	Professional practice: % of patients whose management complied with guidelines Patient: none Seriousness of outcome: MODERATE
Notes	No baseline data
Risk of bias
Bias	Authors' judgement	Support for judgement
Allocation concealment (selection bias)	Low risk	A ‐ Adequate

Chalker 2005.

Methods	Cluster RCT Follow up: providers: DONE patients: N/A Blinded assessment: NOT CLEAR Baseline: DONE Reliable outcomes: NOT CLEAR Protection against contamination: DONE Overall quality: MODERATE
Participants	Two districts were randomly selected from 40 districts in Bangkok to represent each of four types of neighbourhoods (industrial, downtown living, modern living, and suburban), then randomly assigned to the control or intervention group. 78 pharmacies were then randomly selected from the 8 districts and randomly assigned to one of the two groups 78 pharmacies in 8 districts in Bangkok Country: Thailand Proportion of eligible providers who participated: 20% OTHER: pharmacies Academic/Teaching setting; NON‐TEACHING setting Type of targeted behaviour: PRESCRIBING (dispensing of antibiotics and corticosteroids) Complexity of targeted behaviour: MEDIUM
Interventions	1. CME: educational intervention performed in 3 groups. Pharmacy owners and assistants in a 2‐day seminar (case management and rational use of drugs) + enforcement of regulations performed by 6 inspectors + peer review groups 2. No intervention control
Outcomes	Professional practice: % clients receiving steroids at request Patient: none Seriousness of outcome: MODERATE
Notes
Risk of bias
Bias	Authors' judgement	Support for judgement
Allocation concealment (selection bias)	Unclear risk	B ‐ Unclear

Christensen 2004.

Methods	Cluster RCT Follow up: providers: DONE patients: N/A Blinded assessment: DONE Baseline: NOT CLEAR Reliable outcomes: DONE Protection against contamination: DONE Overall quality: MODERATE
Participants	321 general practitioners in 178 practices with 8135 patients Country: Denmark Proportion of eligible providers who participated: 100% General practice Academic/Teaching setting: NOT CLEAR Type of targeted behaviour: Out of hours contacts Complexity of targeted behaviour: LOW
Interventions	1. CME: 5 CME meetings in small groups + economic incentive for a status consultation of patient + feedback/reminder: patients' name and number of contacts once a month 2. No intervention control
Outcomes	Professional practice: none Patient: fall in number of out‐of‐hours service contacts Seriousness of outcome: LOW
Notes	No baseline data
Risk of bias
Bias	Authors' judgement	Support for judgement
Allocation concealment (selection bias)	Unclear risk	B ‐ Unclear

Clark 1998.

Methods	Cluster RCT Follow up: providers: DONE patients: NOT DONE Blinded assessment: DONE Baseline: NOT CLEAR Reliable outcomes: NOT CLEAR Protection against contamination: DONE Overall quality: MODERATE
Participants	74 general practice paediatricians and 637 of their asthma patients Country: USA Proportion of eligible providers who participated: 89% Primary care in paediatrics Academic/Teaching setting: NOT CLEAR Type of targeted behaviour: GENERAL MANAGEMENT OF A PROBLEM (asthma care for children) Complexity of targeted behaviour: MEDIUM
Interventions	1. CME: interactive seminar based on theory of self‐regulation, 5 hrs 2. No intervention control
Outcomes	Professional practice: % parents reporting on some indicators of physician behaviour Patient: indicators of use of care Seriousness of outcome: MODERATE
Notes
Risk of bias
Bias	Authors' judgement	Support for judgement
Allocation concealment (selection bias)	Unclear risk	B ‐ Unclear

Davis 2004.

Methods	Cluster RCT Follow up: providers: DONE patients: NOT DONE Blinded assessment: NOT CLEAR Baseline: DONE Reliable outcomes: DONE Protection against contamination: DONE Overall quality: MODERATE
Participants	General practitioners from 68 practices in 53 locations with 1133 of their patients Country: Scotland Proportion of eligible providers who participated: 91% General practice Academic/Teaching setting: NOT CLEAR Type of targeted behaviour: GENERAL MANAGEMENT OF A PROBLEM (epilepsy care) Complexity of targeted behaviour: MEDIUM
Interventions	1. CME intensive: postal dissemination of guideline + interactive, accredited workshops + dedicated structured protocol documents (tool to be used in patient treatment) + the services of a nurse specialist in epilepsy: who offered advice and training to practices in establishing epilepsy review programs, promoted the use of the guideline in epilepsy management and provided information on epilepsy for both practitioners and patients 2. CME intermediate: postal dissemination of guideline + interactive, accredited workshops + dedicated structured protocol documents (tool to be used in patient treatment) 3. Postal dissemination of a nationally developed guideline
Outcomes	Professional practice: process of care data Patient: SF‐36 general health related quality of life measures Seriousness of outcome: HIGH
Notes	Data on professional outcomes could not be extracted, only patient outcomes
Risk of bias
Bias	Authors' judgement	Support for judgement
Allocation concealment (selection bias)	Low risk	A ‐ Adequate

de Almeida Neto 2000.

Methods	Provider RCT Follow up: providers: DONE patients: N/A Blinded assessment: NOT CLEAR Baseline: DONE Reliable outcomes: NOT CLEAR Protection against contamination: DONE Overall quality: MODERATE
Participants	24 pharmacists in 24 pharmacies Country: Australia Proportion of eligible providers who participated: NOT CLEAR Community‐based care Academic/Teaching setting: NOT CLEAR Type of targeted behaviour: PRESCRIBING (inappropriate drug use) Complexity of targeted behaviour: LOW
Interventions	1. CME: interactive and didactic seminar presenting a pharmacy‐based protocol based on the Stages of change model with practicing of skills, 3 hrs 2. No intervention control
Outcomes	Professional practice: pharmacist behaviour observed by pseudo‐patrons Patient: none
Notes
Risk of bias
Bias	Authors' judgement	Support for judgement
Allocation concealment (selection bias)	Unclear risk	B ‐ Unclear

Delvaux 2005.

Methods	Provider RCT Follow up: providers: NOT DONE patients: N/A Blinded assessment: DONE Baseline: DONE Reliable outcomes: DONE Protection against contamination: NOT CLEAR Overall quality: MODERATE
Participants	72 specialists in medical, surgical oncology, radiotherapy, hematology, gynecology etc, caring for cancer patients Country: Belgium Proportion of eligible providers who participated: 2% Hospital setting, outpatients Academic/Teaching setting: NON‐TEACHING Type of targeted behaviour: COMMUNICATION SKILLS (cancer) Complexity of targeted behaviour: MEDIUM
Interventions	1. CME: 19‐hrs basic training + 3hrs x 6 consolidation small group workshops over 3 months 2. 19‐hrs basic training
Outcomes	Professional practice: patients' satisfaction score with interview Patient: none Seriousness of outcome: HIGH
Notes
Risk of bias
Bias	Authors' judgement	Support for judgement
Allocation concealment (selection bias)	Unclear risk	B ‐ Unclear

Dietrich 1992.

Methods	Cluster RCT Follow up: providers: DONE patients: N/A Blinded assessment: NOT CLEAR Baseline: DONE Reliable outcomes: NOT CLEAR Protection against contamination: DONE Overall quality: MODERATE
Participants	98 doctors in 98 practices providing cancer screening for 2595 patients Country: USA Proportion of eligible providers who participated: NOT CLEAR Primary care Academic/Teaching setting: NON‐TEACHING Type of targeted behaviour: SCREENING (cancer) Complexity of targeted behaviour: MEDIUM
Interventions	1. CME: 1‐day small group discussions of relevant topics 2. 1‐day small group discussions of relevant topics + office system with facilitator 3. Office system with facilitator 4. No intervention control
Outcomes	Professional practice: different cancer screening initiatives measured by patient surveys and chart reviews Patient: none Seriousness of outcome: MODERATE
Notes
Risk of bias
Bias	Authors' judgement	Support for judgement
Allocation concealment (selection bias)	Low risk	A ‐ Adequate

Dolan1997.

Methods	Cluster RCT Follow up: providers: NOT DONE patients: N/A Blinded assessment: NOT CLEAR Baseline: DONE Reliable outcomes: NOT CLEAR Protection against contamination: NOT CLEAR Overall quality: LOW
Participants	82 internal medicine housestaff and attending physicians Country: USA Proportion of eligible providers who participated: NOT CLEAR Hospital, outpatients Academic/Teaching setting: UNIVERSITY BASED Type of targeted behaviour: SCREENING (cancer) Complexity of targeted behaviour: LOW
Interventions	1. CME: educational workshop 1hr x 2 2. No intervention control
Outcomes	Professional practice: median mean proportion of moderate to high risk patients per physician reporting skin cancer control practices Patient: none Seriousness of outcome: MODERATE
Notes
Risk of bias
Bias	Authors' judgement	Support for judgement
Allocation concealment (selection bias)	Unclear risk	B ‐ Unclear

Eaton 1999.

Methods	Cluster RCT Follow up: providers: NOT CLEAR patients: N/A Blinded assessment: DONE Baseline: NOT DONE Reliable outcomes: DONE Protection against contamination: DONE Overall quality: MODERATE
Participants	1 doctor and 1 nurse from each of 30 primary care practices Country: New Zealand Proportion of eligible providers who participated: 10% General practice Academic/Teaching setting: NOT CLEAR Type of targeted behaviour: SPIROMETRY USE (screening, diagnosis and monitoring of respiratory disease) Complexity of targeted behaviour: LOW
Interventions	1. CME: 2 hr workshop: theoretical + practical aspects of performance + handheld spirometer received 2. Handheld spirometer received
Outcomes	Professional practice: spirometry quality assurance data Patient: none Seriousness of outcome: MODERATE
Notes	No baseline data
Risk of bias
Bias	Authors' judgement	Support for judgement
Allocation concealment (selection bias)	Unclear risk	B ‐ Unclear

Engers 2005.

Methods	Cluster RCT Follow up: providers: NOT DONE patients: NOT DONE Blinded assessment: NOT CLEAR Baseline: NOT DONE Reliable outcomes: NOT CLEAR Protection against contamination: DONE Overall quality: LOW
Participants	67 general practitioners with 443 low back pain patients Country: Netherlands Proportion of eligible providers who participated: NOT CLEAR General practice Academic/Teaching setting: NOT CLEAR Type of targeted behaviour: GENERAL MANAGEMENT OF A PROBLEM (low back pain) Complexity of targeted behaviour: LOW
Interventions	1. CME: 2 hr educational session + tools for colloboration with manual, exercise and physical therapists + two scientific articles + national guideline + patient education card 2. No intervention control
Outcomes	Professional practice: % referral to a therapist; prescription of pain medication Patient: % still having low back pain after 6 weeks Seriousness of outcome: LOW
Notes	No baseline data
Risk of bias
Bias	Authors' judgement	Support for judgement
Allocation concealment (selection bias)	Low risk	A ‐ Adequate

Fallowfield 2002.

Methods	Cluster RCT Follow up: providers: DONE patients:DONE Blinded assessment: DONE Baseline: NOT CLEAR Reliable outcomes: NOT CLEAR Protection against contamination: NOT CLEAR Overall quality: MODERATE
Participants	160 oncologists from 34 cancer centres Country: UK Proportion of eligible providers who participated: 80% Outpatient clinic Academic/Teaching setting: NOT CLEAR Type of targeted behaviour: GENERAL MANAGEMENT OF A PROBLEM (communicating with cancer patients) Complexity of targeted behaviour: LOW
Interventions	1. CME: 3‐day residential small group course targeting communication skills + 6 patient simulators to provide feedback 2. Residential course + written feedback 3. Written feedback only 4. No intervention control
Outcomes	Professional practice: counts of communication behaviours Patient: none Seriousness of outcome: MODERATE
Notes	Data unextractable
Risk of bias
Bias	Authors' judgement	Support for judgement
Allocation concealment (selection bias)	Low risk	A ‐ Adequate

Feldman 2004.

Methods	Cluster RCT Follow up: providers: DONE patients: NOT CLEAR Blinded assessment: DONE Baseline: NOT DONE Reliable outcomes: DONE Protection against contamination: NOT CLEAR Overall quality: MODERATE
Participants	205 nurses in a large urban home health care agency and their 371 Medicare congestive heart failure patients Country: USA Proportion of eligible providers who participated: NOT CLEAR Community‐based care Academic/Teaching setting: NOT CLEAR Type of targeted behaviour: GENERAL MANAGEMENT OF A PROBLEM (care for patients with heart failure) Complexity of targeted behaviour: HIGH
Interventions	1. CME: interactive practitioner training with role‐play and audiotaping 2. No intervention control
Outcomes	Professional practice: mean number of skilled nursing visits delivered within 90 days Patient: none Seriousness of outcome: HIGH
Notes	No baseline data
Risk of bias
Bias	Authors' judgement	Support for judgement
Allocation concealment (selection bias)	Unclear risk	B ‐ Unclear

Figueiras 2001.

Methods	Cluster RCT Follow up: providers: NOT CLEAR patients: N/A Blinded assessment: DONE Baseline: DONE Reliable outcomes: DONE Protection against contamination: DONE Overall quality: MODERATE
Participants	595 general practitioners in 15 geographical areas Country: Spain Proportion of eligible providers who participated: NOT CLEAR General practice Academic/Teaching setting: NON‐TEACHING Type of targeted behaviour: PRESCRIBING (improvement of prescription of NSAIDS) Complexity of targeted behaviour: LOW
Interventions	1. Educational outreach 2. CME: 45‐min educational session 3. No intervention control
Outcomes	Professional practice: prescription of non‐steroidal NSAIDS as first choice Patient: none Seriousness of outcome: LOW
Notes
Risk of bias
Bias	Authors' judgement	Support for judgement
Allocation concealment (selection bias)	Low risk	A ‐ Adequate

Flottorp 2002.

Methods	Cluster RCT Follow up: providers: DONE patients: N/A Blinded assessment: DONE Baseline: DONE Reliable outcomes: DONE Protection against contamination: DONE Overall quality: HIGH
Participants	Approximately 650 general practitioners in 142 practices Country: Norway Proportion of eligible providers who participated: 49% General practice Academic/Teaching setting: NON‐TEACHING Type of targeted behaviour: GENERAL MANAGEMENT OF A PROBLEM (urinary tract infections in women and sore throat) Complexity of targeted behaviour: LOW
Interventions	1. CME: 1‐day interactive course about urinary tract infections + summary of the main recommendations in electronic and poster format + patient educational material in electronic and paper format + compute‐based decision support and reminders during consultations + increase in the fee for telephone consultations + printed material to facilitate discussions in the practice + points in the continuing medical education programme of The Norwegian Medical Association 2. 1‐day interactive course about sore throat + summary of the main recommendations in electronic and poster format + patient educational material in electronic and paper format + computer‐based decision support and reminders during consultations + increase in the fee for telephone consultations + printed material to facilitate discussions in the practice + points in the continuing medical education programme of The Norwegian Medical Association
Outcomes	Professional practice: use of laboratory tests Patient: None Seriousness of outcome: LOW
Notes
Risk of bias
Bias	Authors' judgement	Support for judgement
Allocation concealment (selection bias)	Low risk	A ‐ Adequate

Forsetlund 2003.

Methods	Provider RCT Follow up: providers: DONE patients: N/A Blinded assessment: NOT CLEAR Baseline: NOT DONE Reliable outcomes: NOT CLEAR Protection against contamination: DONE Overall quality: MODERATE
Participants	148 public health physicians Country: Norway Proportion of eligible providers who participated: 45% General practice Academic/Teaching setting: NON‐TEACHING Type of targeted behaviour: GENERAL MANAGEMENT OF A PROBLEM (working evidence‐based in a public health practice) Complexity of targeted behaviour: HIGH
Interventions	1. CME: 1 ‐ 5 day workshop: (1 day: 10 physicians; 3 days: 21 physicians; 5 days: 18 physicians) + discussion list + help desk and information service + access to 5 databases + 3 newsletters
Outcomes	Professional practice: % of physicians having used research explicitly to some degree (used/not used) Patient: none Seriousness of outcome: MODERATE
Notes
Risk of bias
Bias	Authors' judgement	Support for judgement
Allocation concealment (selection bias)	Low risk	A ‐ Adequate

Garcia 2003.

Methods	Cluster RCT Follow up: providers: DONE patients: N/A Blinded assessment: DONE Baseline: NOT DONE Reliable outcomes: NOT CLEAR Protection against contamination: DONE Overall quality: MODERATE
Participants	14 districts of low socioeconomic status in Lima (pharmacies and physicians) Country: Peru Proportion of eligible providers who participated: 79% Setting of care: pharmacies Academic/Teaching setting: NON‐TEACHING Type of targeted behaviour: GENERAL MANAGEMENT OF A PROBLEM (recognition, management and prevention of STDs) Complexity of targeted behaviour: HIGH
Interventions	1. CME: 1.5 hrs x 3=4.5‐hr luncheon training seminars + physicians in each district invited to attend a 6 hr workshop on management of STD syndromes + referral network + monthly follow‐up visits for six months to all certified pharmacies and referral physicians and health centres within their district. 2. Seminar on diarrhoea
Outcomes	Professional practice: recognition of STD symptoms, adequate management of the syndrome, recommendations for use of condoms and recommendations for treatment of partners Patient: none Seriousness of outcome: HIGH
Notes
Risk of bias
Bias	Authors' judgement	Support for judgement
Allocation concealment (selection bias)	Unclear risk	B ‐ Unclear

Gask 2004.

Methods	Cluster RCT Follow up: providers: DONE patients: NOT DONE Blinded assessment: DONE Baseline: NOT DONE Reliable outcomes: DONE Protection against contamination: DONE Overall quality: MODERATE
Participants	38 general practitioners with 189 patients Country: UK Proportion of eligible providers who participated: 7% Setting of care: general practice Academic/Teaching setting: NON‐TEACHING Type of targeted behaviour: GENERAL MANAGEMENT OF A PROBLEM (assessment and management of depression) Complexity of targeted behaviour: MEDIUM
Interventions	1. CME: 2 hr x 5 approved training course sessions, including role play 2. No intervention control
Outcomes	Professional practice: 7 indicators for patient satisfaction with doctor Patient: Hamilton depression score Seriousness of outcome: MODERATE
Notes	No post‐test data
Risk of bias
Bias	Authors' judgement	Support for judgement
Allocation concealment (selection bias)	Low risk	A ‐ Adequate

Gerrity 1999.

Methods	Provider RCT Follow up: providers: DONE patients: N/A Blinded assessment: DONE Baseline: NOT DONE Reliable outcomes: NOT CLEAR Protection against contamination: NOT CLEAR Overall quality: MODERATE
Participants	49 primary care physicians (two standardized simulated patients) Country: UK Proportion of eligible providers who participated: NOT CLEAR Setting of care: General practice Academic/Teaching setting: NON‐TEACHING Type of targeted behaviour: GENERAL MANAGEMENT OF A PROBLEM (depression) Complexity of targeted behaviour: MEDIUM
Interventions	1. CME: 4hr x 2 education sessions given two weeks apart with guidelines + goal‐setting + doing a videotape of a patient interview as homework for discussion at last session 2. No intervention control
Outcomes	Professional practice: physicians' behaviour as reported by standardised patients Patient: None Seriousness of outcome: MODERATE
Notes	No baseline data
Risk of bias
Bias	Authors' judgement	Support for judgement
Allocation concealment (selection bias)	Unclear risk	B ‐ Unclear

Gilroy 2004.

Methods	Cluster RCT Follow up: providers: NOT CLEAR patients: N/A Blinded assessment: NOT CLEAR Baseline: NOT DONE Reliable outcomes: DONE Protection against contamination: DONE Overall quality: LOW
Participants	Head nurses in 10 community health centres Country: Malawi Proportion of eligible providers who participated: NOT CLEAR Setting of care: Community‐based care Academic/Teaching setting: NON‐TEACHING Type of targeted behaviour: GENERAL MANAGEMENT OF A PROBLEM (drug counseling to parents with sick children) Complexity of targeted behaviour: LOW
Interventions	1. CME: 11‐day training course with nationally adapted guidelines + one supervisory visit from course instructors 2. No intervention control
Outcomes	Professional practice: 10‐point composite scale measuring the quality of drug counselling Patient: none Seriousness of outcome: MODERATE
Notes
Risk of bias
Bias	Authors' judgement	Support for judgement
Allocation concealment (selection bias)	Unclear risk	B ‐ Unclear

Goldberg 2001.

Methods	Cluster RCT Follow up: providers: NOT CLEAR patients: DONE Blinded assessment: DONE Baseline: DONE Reliable outcomes: DONE Protection against contamination: DONE Overall quality: MODERATE
Participants	Spine surgeons, primary care physicians, patients who were surgical candidates, and hospital administrators in ten communities with annual rates of back surgery above the 1990 national average Country: USA Proportion of eligible providers who participated: 12% Setting of care: hospital setting Academic/Teaching setting: NOT CLEAR Type of targeted behaviour: Surgery (rate of back surgery) Complexity of targeted behaviour: HIGH
Interventions	1. CME: regional study group meetings for neurosurgeons and orthopedists + CME conferences for primary care providers + mailed generalist academic detailing + videodisc patient decision making + small discussion groups of key administrative personnel 2. No intervention control
Outcomes	Professional practice: low‐back surgical rate Patient: none Seriousness of outcome: HIGH
Notes
Risk of bias
Bias	Authors' judgement	Support for judgement
Allocation concealment (selection bias)	Unclear risk	B ‐ Unclear

Gray 2004.

Methods	Cluster RCT Follow up: providers: DONE patients: NOT DONE Blinded assessment: DONE Baseline: NOT CLEAR Reliable outcomes: DONE Protection against contamination: DONE Overall quality: MODERATE
Participants	12 clusters of 5 community mental health nurses each, based on geographical location Country: UK Proportion of eligible providers who participated: NOT CLEAR Setting of care: community‐based care Academic/Teaching setting: NON‐TEACHING Type of targeted behaviour: GENERAL MANAGEMENT OF A PROBLEM (compliance therapy) Complexity of targeted behaviour: HIGH
Interventions	1. CME: 80 hrs of teaching delivered on a day‐release basis over 10 weeks 2. No intervention control
Outcomes	Professional practice: none Patient: compliance with schizophrenia medication Seriousness of outcome: HIGH
Notes	Cost of training each community mental health nurse: £ 1474
Risk of bias
Bias	Authors' judgement	Support for judgement
Allocation concealment (selection bias)	Low risk	A ‐ Adequate

Gülmezoglu 2006.

Methods	Cluster RCT Follow up: providers: DONE patients: N/A Blinded assessment: DONE Baseline: DONE Reliable outcomes: DONE Protection against contamination: DONE Overall quality: HIGH
Participants	Doctors, midwives, interns and students in obstetric practices in 22 hospitals in Mexico City and 18 in Thailand Country: Mexico and Thailand Proportion of eligible providers who participated: 65% Hospital, inpatients Academic/Teaching setting: NON‐TTEACHING Type of targeted behaviour: GENERAL MANAGEMENT OF A PROBLEM (OBSTETRIC PRACTICES) Complexity of targeted behaviour: HIGH
Interventions	1. CME: series of 3 workshops at time 0, after 6 weeks and after 6 months + Meeting with hospital directors and department heads + Provision of the database, computers and printers + coordinator from each hospital 2. No intervention control
Outcomes	Professional practice: % change in practice rates in 6 provider behaviours for obstetric care Patient: none Seriousness of outcome: HIGH
Notes
Risk of bias
Bias	Authors' judgement	Support for judgement
Allocation concealment (selection bias)	Low risk	A ‐ Adequate

Hadiyono 1996.

Methods	Cluster RCT Follow up: providers: DONE patients: N/A Blinded assessment: DONE Baseline: DONE Reliable outcomes: NOT CLEAR Protection against contamination: NOT CLEAR Overall quality: HIGH
Participants	Prescribers in 24 health centres. Country: Indonesia Proportion of eligible providers who participated: 83% Community‐based care Academic/Teaching setting: NOT CLEAR Type of targeted behaviour: PRESCRIBING (reduce the use of injections) Complexity of targeted behaviour: LOW
Interventions	1. CME: Educational workshop 1.5 ‐ 2 hrs with 6 prescribers and 6 patients/community members to discuss reasons for injection used and to arrive at a consensus 2. No intervention control
Outcomes	Professional practice: reduction of injection use Patient: none Seriousness of outcome: LOW
Notes
Risk of bias
Bias	Authors' judgement	Support for judgement
Allocation concealment (selection bias)	Low risk	A ‐ Adequate

Harmsen 2005.

Methods	Provider RCT Follow up: providers: DONE patients: NOT CLEAR Blinded assessment: DONE Baseline: NOT CLEAR Reliable outcomes: NOT CLEAR Protection against contamination: DONE Overall quality: MODERATE
Participants	38 general practitioners with a practice population of at least 25% of non‐Western country of origin, resulting in a total of 986 consultations Country: Netherlands Proportion of eligible providers who participated: 22% General practice Academic/Teaching setting: NON‐TEACHING Type of targeted behaviour: GENERAL MANAGEMENT OF A PROBLEM (intercultural communication skills) Complexity of targeted behaviour: LOW
Interventions	1. CME, physician intervention: 2.5‐day training on intercultural communication based on Pinto's 'three‐step method' + patient intervention: 12‐min videotaped instruction in the waiting room that the patient should feel free to communicate directly and express any disagreement 2. No intervention control
Outcomes	Professional practice: one indicator for mutual understanding and three indicators for patients' satisfaction Patient: none Seriousness of outcome: HIGH
Notes
Risk of bias
Bias	Authors' judgement	Support for judgement
Allocation concealment (selection bias)	Unclear risk	B ‐ Unclear

Heale 1988.

Methods	Provider RCT Follow up: providers: NOT DONE patients: N/A Blinded assessment: NOT CLEAR Baseline: NOT DONE Reliable outcomes: NOT CLEAR Protection against contamination: NOT CLEAR Overall quality: LOW
Participants	46 family doctors, providing care for patients with one of 6 common problems: transient ischaemic attacks, hypertension, pre‐menstrual syndrome, chlamydial infections, dementia, prescribing Country: Canada Proportion of eligible providers who participated: 52% Family practice Academic/Teaching setting: NOT CLEAR Type of targeted behaviour: GENERAL MANAGEMENT OF A PROBLEM (clinical problems within family medicine) Complexity of targeted behaviour: MEDIUM
Interventions	1. CME: small group, problem‐based sessions 2. CME: large group, case problem discussion 3. CME: traditional didactic lecture
Outcomes	Professional practice: performance score as rated by simulated patient visit Patient: none Seriousness of outcome: LOW
Notes
Risk of bias
Bias	Authors' judgement	Support for judgement
Allocation concealment (selection bias)	Unclear risk	B ‐ Unclear

Heller 2001.

Methods	Cluster RCT Follow up: providers: DONE patients: N/A Blinded assessment: DONE Baseline: NOT CLEAR Reliable outcomes: NOT CLEAR Protection against contamination: DONE Overall quality: MODERATE
Participants	Doctors and allied health staff in 37 hospitals with 3240 patients Country: UK Proportion of eligible providers who participated: NOT CLEAR Hospital Academic/Teaching setting: NOT CLEAR Type of targeted behaviour: GENERAL MANAGEMENT OF A PROBLEM (management of unstable angina) Complexity of targeted behaviour: HIGH
Interventions	1. CME: educational session run by a local opinion leader, including feedback on hospital level 2. No intervention control
Outcomes	Professional practice: % compliance with guideline for angina Patient: none Seriousness of outcome: HIGH
Notes
Risk of bias
Bias	Authors' judgement	Support for judgement
Allocation concealment (selection bias)	Unclear risk	B ‐ Unclear

Jennett 1988.

Methods	Cluster RCT Follow up: providers: DONE patients: N/A Blinded assessment: DONE Baseline: DONE Reliable outcomes: DONE Protection against contamination: DONE Overall quality: HIGH
Participants	31 family doctors in 25 practices providing care for 2077 episodes of patients with risk of colorectal or prostatic cancer or with hypertension Country: Canada Proportion of eligible providers who participated: 12% Community‐based care Academic/Teaching setting: NOT CLEAR Type of targeted behaviour: GENERAL MANAGEMENT OF A PROBLEM (cancer screening and hypertension management) Complexity of targeted behaviour: LOW
Interventions	1. CME: 1.5‐hour small group meeting + 2 teleconferences over 6‐8 weeks + 4 newsletters in cancer screening 2. CME: 1.5‐hour small group meeting + 2 teleconferences over 6‐8 weeks + 4 newsletters in hypertension management 2. No intervention control
Outcomes	Professional practice: proportion of recommended behaviours in cancer screening and hypertension management Patient: none Seriousness of outcome: MODERATE
Notes
Risk of bias
Bias	Authors' judgement	Support for judgement
Allocation concealment (selection bias)	Low risk	A ‐ Adequate

Jones 1998.

Methods	Cluster RCT Follow up: providers: NOT DONE patients: DONE Blinded assessment: NOT DONE Baseline: DONE Reliable outcomes: NOT CLEAR Protection against contamination: NOT CLEAR Overall quality: LOW
Participants	116 nurses in 6 wards in two hospitals Country: UK Proportion of eligible providers who participated: NOT CLEAR Hospital: stroke units and wards Academic/Teaching setting: UNIVERSITY BASED Type of targeted behaviour: REHABILITATION (of stroke patients) Complexity of targeted behaviour: HIGH
Interventions	1. CME: mixed teaching format lessons 2hr x 2 2. No intervention control
Outcomes	Professional practice: % correct positions in a set of observations Patient: none Seriousness of outcome: HIGH
Notes
Risk of bias
Bias	Authors' judgement	Support for judgement
Allocation concealment (selection bias)	Low risk	A ‐ Adequate

Kasje 2004.

Methods	Cluster RCT Follow up: providers: NOT DONE patients: NOT DONE Blinded assessment: DONE Baseline: NOT DONE Reliable outcomes: DONE Protection against contamination: DONE Overall quality: MODERATE
Participants	245 general practitioners in 27 peer review groups Country: Netherlands Proportion of eligible providers who participated: 93% General practice Academic/Teaching setting: NON‐TEACHING Type of targeted behaviour: PRESCRIBING (chronic heart failure and diabetes mellitus type 2) Complexity of targeted behaviour: LOW
Interventions	1a. CME: 13 peer review groups in arm for condition chronic heart failure: 1‐hr interactive peer group session on management of condition with case based discussions 2a. No intervention control 1b. CME: 14 peer review groups in arm for condition diabetes mellitus type 2: 1‐hr interactive peer group session on management of condition with case based discussions 2b. No intervention control
Outcomes	Professional practice: % of patients receiving prescription of ACE inhibitors Patient: none Seriousness of outcome: MODERATE
Notes
Risk of bias
Bias	Authors' judgement	Support for judgement
Allocation concealment (selection bias)	Low risk	A ‐ Adequate

Kerse 1999.

Methods	Cluster RCT Follow up: providers: DONE patients: NOT DONE Blinded assessment: DONE Baseline: NOT CLEAR Reliable outcomes: NOT CLEAR Protection against contamination: DONE Overall quality: MODERATE
Participants	42 general practitioners and 267 of their patients aged over 65 years Country: Australia Proportion of eligible providers who participated: 51% general practice Academic/Teaching setting: NOT CLEAR Type of targeted behaviour: PREVENTIVE CARE (health promotion for elderly people) Complexity of targeted behaviour: MEDIUM
Interventions	1. CME: 3‐hr didactic seminar on health issues + audit & feedback + 15 mins outreach + card‐based prompt system + resource directory 2. None
Outcomes	Professional practice: patients' recall of discussions with the general practitioner Patient: several health outcomes Seriousness of outcome: MODERATE
Notes	No baseline data
Risk of bias
Bias	Authors' judgement	Support for judgement
Allocation concealment (selection bias)	Low risk	A ‐ Adequate

Kiessling 2002.

Methods	Cluster RCT Follow up: providers: DONE patients: DONE Blinded assessment: DONE Baseline: DONE Reliable outcomes: DONE Protection against contamination: DONE Overall quality: HIGH
Participants	54 general practitioners and 88 of their patients with coronary artery disease Country: Sweden Proportion of eligible providers who participated: NOT CLEAR General practice Academic/Teaching setting: NON‐TEACHING Type of targeted behaviour: PREVENTIVE CARE (coronary artery disease) Complexity of targeted behaviour: LOW
Interventions	1. CME: guidelines distributed and presented in a lecture + 1 hr x 3 (4) of case‐based education 2. Guidelines distributed and presented in a lecture
Outcomes	Professional practice: none (only self‐reported) Patient: low density lipoprotein cholesterol (mmol/l) Seriousness of outcome: MODERATE
Notes
Risk of bias
Bias	Authors' judgement	Support for judgement
Allocation concealment (selection bias)	Low risk	A ‐ Adequate

Kimberlin 1993.

Methods	Cluster RCT Follow up: providers: DONE patients: NOT CLEAR Blinded assessment: NOT CLEAR Baseline: NOT DONE Reliable outcomes: DONE Protection against contamination: DONE Overall quality: MODERATE
Participants	102 community‐based pharmacists providing prescriptions for 762 elderly patients Country: USA Proportion of eligible providers who participated: 24% Community‐based care Academic/Teaching setting: NON‐TEACHING Type of targeted behaviour: PRESCRIBING (detection of drug problems) Complexity of targeted behaviour: MEDIUM
Interventions	1. CME: homework with post‐test + 1‐day seminar (mixed format) + follow‐up service (help desk) 2. No intervention control
Outcomes	Professional practice: % of patients reporting positively about pharmacists' counselling about prescriptions (7 behaviours) Patient: none Seriousness of outcome: MODERATE
Notes	No baseline data
Risk of bias
Bias	Authors' judgement	Support for judgement
Allocation concealment (selection bias)	Unclear risk	B ‐ Unclear

King 2002.

Methods	Cluster RCT Follow up: providers: NOT DONE patients: NOT CLEAR Blinded assessment: NOT CLEAR Baseline: DONE Reliable outcomes: DONE Protection against contamination: DONE Overall quality: LOW
Participants	84 general practitioner principals and 272 patients attending their practices who scored above the threshold for psychological distress Country: UK Proportion of eligible providers who participated: 10% General practice Academic/Teaching setting: NOT CLEAR Type of targeted behaviour: GENERAL MANAGEMENT OF A PROBLEM (brief cognitive therapy for depression) Complexity of targeted behaviour: MEDIUM
Interventions	1. CME: 4 half‐day training sessions with introduction of guideline for cognitive behaviour therapy 2. No intervention control
Outcomes	Professional practice: (physicians' self reported atttitudes and knowledge) Patient: patients' score on anxiety scales, Beck inventory and SF‐36 dimensions Seriousness of outcome: MODERATE
Notes
Risk of bias
Bias	Authors' judgement	Support for judgement
Allocation concealment (selection bias)	Unclear risk	B ‐ Unclear

Kottke 1989.

Methods	Cluster RCT Follow up: providers: DONE patients: NOT CLEAR Blinded assessment: NOT CLEAR Baseline: NOT DONE Reliable outcomes: NOT CLEAR Protection against contamination: DONE Overall quality: LOW
Participants	66 doctors general/family practices providing smoking cessation interventions for 1653 patients Country: USA Proportion of eligible providers who participated: 6% General practice Academic/Teaching setting: NOT CLEAR Type of targeted behaviour: PREVENTIVE CARE (smoking counselling) Complexity of targeted behaviour: MEDIUM
Interventions	1. CME: 6‐hr workshop of mixed format + 100 copies of 'Quit and win' smoking cessation manual for patients 2. 100 copies of 'Quit and win' smoking cessation manual for patients 3. No intervention control
Outcomes	Professional practice: patients' reports of smoking cessation counselling behaviours Patient: number of patients smoking after 1 year Seriousness of outcome: MODERATE
Notes
Risk of bias
Bias	Authors' judgement	Support for judgement
Allocation concealment (selection bias)	Unclear risk	D ‐ Not used

Levinson 1993.

Methods	Cluster RCT Follow up: providers: DONE patients: N/A Blinded assessment: DONE Baseline: NOT CLEAR Reliable outcomes: NOT DONE Protection against contamination: NOT CLEAR Overall quality: MODERATE
Participants	31 general internists, family doctors in practices encouraged to improve communications skills for 473 patients Country: USA Proportion of eligible providers who participated: NOT CLEAR General practice Academic/Teaching setting: NOT CLEAR Type of targeted behaviour: DOCTOR ‐ PATIENT COMMUNICATION Complexity of targeted behaviour: MEDIUM
Interventions	1. CME: 4.5‐hour didactic presentation + case‐based discussion 2. No intervention control
Outcomes	Professional practice: indicators for communication skills with patients in primary care (mean number of statements) Patient: none Seriousness of outcome: MODERATE
Notes	No baseline data
Risk of bias
Bias	Authors' judgement	Support for judgement
Allocation concealment (selection bias)	Unclear risk	B ‐ Unclear

Leviton 1999.

Methods	Cluster RCT Follow up: providers: DONE patients: N/A Blinded assessment: NOT CLEAR Baseline: DONE Reliable outcomes: DONE Protection against contamination: DONE Overall quality: HIGH
Participants	Estimated 405 obstetricians in 27 hospitals and their preterm delivery cases Country: UK Proportion of eligible providers who participated: 90% Community‐based care Academic/Teaching setting: MIXED Type of targeted behaviour: GENERAL MANAGEMENT OF A PROBLEM (preterm delivery) Complexity of targeted behaviour: MEDIUM
Interventions	1. CME: opinion leaders (one physician and one nurse) appointed from each hospital by the director to serve as local experts + grand rounds lecture on antenatal corticosteroid therapy given by a nationally respected expert + chart reminder system for prompting for therapy + 1‐hour group discussion with scenarios, led by opinion leaders + monitoring care to provide feedback 2. No intervention control
Outcomes	Professional practice: % of patients receiving antenatal corticosteroids Patient: none Seriousness of outcome: HIGH
Notes
Risk of bias
Bias	Authors' judgement	Support for judgement
Allocation concealment (selection bias)	Low risk	A ‐ Adequate

Lewin 2005.

Methods	Cluster RCT Follow up: providers: DONE patients: N/A Blinded assessment: DONE Baseline: NOT CLEAR Reliable outcomes: DONE Protection against contamination: DONE Overall quality: HIGH
Participants	Clinic staff in 24 nurse‐managed municipal primary health clinics with treatment succes rates below 70% Country: South Africa Proportion of eligible providers who participated: 62% Community‐based care Academic/Teaching setting: NON‐TEACHING Type of targeted behaviour: GENERAL MANAGEMENT OF A PROBLEM (tuberculosis) Complexity of targeted behaviour: MEDIUM
Interventions	1. CME: 7 training modules each constituting one training session of 3 hrs + homework + meetings and telephone discussions, drawing on a number of theoretical models 2. No intervention control
Outcomes	Professional practice: none Patient: % of patients with successfull TB treatment completion Seriousness of outcome: HIGH
Notes
Risk of bias
Bias	Authors' judgement	Support for judgement
Allocation concealment (selection bias)	Low risk	A ‐ Adequate

Maiman 1988.

Methods	Cluster RCT Follow up: providers: DONE patients: DONE Blinded assessment: NOT DONE Baseline: NOT CLEAR Reliable outcomes: NOT CLEAR Protection against contamination: NOT DONE Overall quality: LOW
Participants	83 paediatricians in practices, encouraged to provide medication compliance strategies to patients with otitis media Country: USA Proportion of eligible providers who participated: 94% General practice Academic/Teaching setting: NOT CLEAR Type of targeted behaviour: PRESCRIBING (compliance enhancing practices) Complexity of targeted behaviour: LOW
Interventions	1. CME: tutorial and accompanying printed materials: 2.5 x 2‐hr didactic and group discussion + education material 2. Mailed printed materials 3. No intervention control
Outcomes	Professional practice: compliance‐enhancing strategies and patients with no missed doses Patient: none Seriousness of outcome: LOW
Notes	No baseline data
Risk of bias
Bias	Authors' judgement	Support for judgement
Allocation concealment (selection bias)	Unclear risk	B ‐ Unclear

Mazzuca 1987.

Methods	Cluster RCT Follow up: providers: DONE patients: N/A Blinded assessment: DONE Baseline: DONE Reliable outcomes: NOT CLEAR Protection against contamination: DONE Overall quality: MODERATE
Participants	29 public health nurses in seven older‐adult clinics providing care for patients with arthritis Country: USA Proportion of eligible providers who participated: NOT CLEAR Community‐based care Academic/Teaching setting: NON‐TEACHING Type of targeted behaviour: Screening and management (of arthritis in elderly patients) Complexity of targeted behaviour: LOW
Interventions	1. CME: inservice education program on arthritis screening and management in older adults: distribution of required readings; 3hrs mixed format + help desk + laminated screening and management guide + telephone consultation service + 1 hr individual skills training 2. No intervention control
Outcomes	Professional practice: % of clients screened for joint swelling Patient: none Seriousness of outcome: MODERATE
Notes
Risk of bias
Bias	Authors' judgement	Support for judgement
Allocation concealment (selection bias)	Low risk	A ‐ Adequate

Meyer 2001.

Methods	Cluster RCT Follow up: providers: NOT CLEAR patients: N/A Blinded assessment: NOT CLEAR Baseline: NOT CLEAR Reliable outcomes: NOT CLEAR Protection against contamination: DONE Overall quality: MODERATE
Participants	Primary health care nurses in 22 primary health care clinics Country: South Africa Proportion of eligible providers who participated: 51% Community‐based care Academic/Teaching setting: NOT CLEAR Type of targeted behaviour: PRESCRIBING (prescribing practices) Complexity of targeted behaviour: MEDIUM
Interventions	1. CME: 4 days problem‐based educational sessions using material from the WHO's '"Guide to Good Prescribing'" 2. No intervention control
Outcomes	Professional practice: indicators for rational prescribing in respiratory tract infection and diarrhoea/vomiting Patient: none Seriousness of outcome: HIGH
Notes
Risk of bias
Bias	Authors' judgement	Support for judgement
Allocation concealment (selection bias)	Low risk	A ‐ Adequate

Moore 2003 a.

Methods	Cluster RCT Follow up: providers: NOT CLEAR patients: DONE Blinded assessment: DONE Baseline: NOT CLEAR Reliable outcomes: NOT CLEAR Protection against contamination: DONE Overall quality: MODERATE
Participants	84 providers: general practitioners, health visitors, district nurses, midwives and nurse practitioners in 12 general practices Country: UK Proportion of eligible providers who participated: 23% General practice Academic/Teaching setting: NOT CLEAR Type of targeted behaviour: GENERAL MANAGEMENT OF A PROBLEM (nutrition councelling) Complexity of targeted behaviour: LOW
Interventions	Phase 1: 90‐min x 3 education for each practice: small groups, multidisciplinary general practice teams, conducted by local clinical opinion leader + Phase 2: 90‐min x 2 held on practices' premises focusing on practicing skills (6 months period) + diet sheets and patient teaching aids
Outcomes	Professional practice: % of patients reporting having discussed diet with doctor or 7 indicators for phys' behaviour as reported by patients Patient: none Seriousness of outcome: LOW
Notes	No baseline data
Risk of bias
Bias	Authors' judgement	Support for judgement
Allocation concealment (selection bias)	Low risk	A ‐ Adequate

Moore 2003 b.

Methods	Cluster RCT Follow up: providers: NOT DONE patients: NOT DONE Blinded assessment: DONE Baseline: DONE Reliable outcomes: NOT CLEAR Protection against contamination: DONE Overall quality: MODERATE
Participants	245 general practitioners and practice nurses in 44 practices Country: UK Proportion of eligible providers who participated: 28% General practice Academic/Teaching setting: NOT CLEAR Type of targeted behaviour: GENERAL MANAGEMENT OF A PROBLEM (obesity) Complexity of targeted behaviour: MEDIUM
Interventions	90‐min x 3 education for each practice: small groups, multidisciplinary general practice teams, conducted by dietitians + tool for estimation of a patient's daily energy requirement + diet sheets and supporting written resources to facilitate the dietary prescription to patients
Outcomes	Professional practice: none Patient: difference in mean weight of patients 12 months after the intervention Seriousness of outcome: LOW
Notes
Risk of bias
Bias	Authors' judgement	Support for judgement
Allocation concealment (selection bias)	Low risk	A ‐ Adequate

Morrison 2001.

Methods	Cluster RCT Follow up: providers: NOT CLEAR patients: N/A Blinded assessment: NOT CLEAR Baseline: NOT DONE Reliable outcomes: NOT CLEAR Protection against contamination: DONE Overall quality: LOW
Participants	335 general practitioners from 221 practices with 689 referrals Country: UK Proportion of eligible providers who participated: 50% General practice Academic/Teaching setting: NOT CLEAR Type of targeted behaviour: GENERAL MANAGEMENT OF A PROBLEM (infertility) Complexity of targeted behaviour: MEDIUM
Interventions	1. CME: guideline + invitation to a discussion meeting + invitation to have an individual visit + individual meetings with key personnel to inform about the project 2. Information that a guideline would be received in 12 months
Outcomes	Professional practice: mean number of tests Patient: none Seriousness of outcome: MODERATE
Notes
Risk of bias
Bias	Authors' judgement	Support for judgement
Allocation concealment (selection bias)	Low risk	A ‐ Adequate

Ockene 1996.

Methods	Cluster RCT Follow up: providers: DONE patients: N/A Blinded assessment: NOT CLEAR Baseline: NOT DONE Reliable outcomes: NOT CLEAR Protection against contamination: DONE Overall quality: LOW
Participants	45 internists in a managed care setting, providing nutrition counselling in hyperlipidemia Country: USA Proportion of eligible providers who participated: 98% General practice Academic/Teaching setting: NON‐TEACHING Type of targeted behaviour: PREVENTIVE CARE (nutrition counselling for patients with hyperlipidemia) Complexity of targeted behaviour: LOW
Interventions	1. CME: 2.5‐hr mixed session (role play, didactic) + patient dietary form, followed by 0.5‐hour individualised tutorial 2. 2.5‐hr mixed session (role play, didactic) + patient dietary form, followed by 0.5‐hour individualised tutorial + structured office management system 3. Usual care
Outcomes	Professional practice: indicators for nutrition counselling and referral (patients' responses) Patient: none Seriousness of outcome: LOW
Notes	No baseline data
Risk of bias
Bias	Authors' judgement	Support for judgement
Allocation concealment (selection bias)	Low risk	A ‐ Adequate

Ornstein 2004.

Methods	Cluster RCT Follow up: providers: DONE patients: NOT CLEAR Blinded assessment: DONE Baseline: DONE Reliable outcomes: DONE Protection against contamination: DONE Overall quality: HIGH
Participants	20 community‐based family or general internal medicine practices (44 physicians and 17 midlevel providers) in 14 states Country: USA Proportion of eligible providers who participated: NOT CLEAR General practice Academic/Teaching setting: NON‐TEACHING Type of targeted behaviour: GENERAL MANAGEMENT OF A PROBLEM (preventive cardiovascular care) Complexity of targeted behaviour: MEDIUM
Interventions	1. CME: 2‐day network meetings + feedback report + outreach 2. Feedback reports
Outcomes	Professional practice: % of performance targets achieved Patient: % of clinical targets achieved Seriousness of outcome: MODERATE
Notes
Risk of bias
Bias	Authors' judgement	Support for judgement
Allocation concealment (selection bias)	Low risk	A ‐ Adequate

Parker 1995.

Methods	Cluster RCT Follow up: providers: NOT DONE patients: NOT DONE Blinded assessment: NOT CLEAR Baseline: NOT DONE Reliable outcomes: NOT CLEAR Protection against contamination: DONE Overall quality: LOW
Participants	35 nursing staffs from four long‐term care facilities with a minimum of 20 residents who had been diagnoses with diabetes Country: USA Proportion of eligible providers who participated: 10% Long‐term care facilities Academic/Teaching setting: NON‐TEACHING Type of targeted behaviour: GENERAL MANAGEMENT OF A PROBLEM (improving diabetes care) Complexity of targeted behaviour: LOW
Interventions	1. CME: 20‐min didactic sessions x 7, slides followed by questions and answers 2. No intervention control
Outcomes	Professional practice: 5 indicators of care for patients with diabetes Patient: none Seriousness of outcome: MODERATE
Notes
Risk of bias
Bias	Authors' judgement	Support for judgement
Allocation concealment (selection bias)	Unclear risk	B ‐ Unclear

Pekarik 1994.

Methods	Cluster RCT Follow up: providers: NOT CLEAR patients: NOT DONE Blinded assessment: DONE Baseline: DONE Reliable outcomes: NOT CLEAR Protection against contamination: NOT CLEAR Overall quality: LOW
Participants	22 psychotherapists in 3 outpatient clinics in 3 midwestern cities Country: USA Proportion of eligible providers who participated: NOT CLEAR Outpatient clinics Academic/Teaching setting: NON‐TEACHING Type of targeted behaviour: GENERAL MANAGEMENT OF A PROBLEM (duration of psychotherapy) Complexity of targeted behaviour: MEDIUM
Interventions	1. CME mixed: 1‐day workshop (didactic presentation, skills training, case review, 'homework' given for the follow‐up) + follow‐up: 1.5 hours held once with supervision and discussion of participants' report of a session with a patient that had been selected for brief therapy 2. None
Outcomes	Professional practice: none extracted Patient: scores on the Brief Symptom Inventory Seriousness of outcome: MODERATE
Notes
Risk of bias
Bias	Authors' judgement	Support for judgement
Allocation concealment (selection bias)	Low risk	A ‐ Adequate

Perera 1983.

Methods	Cluster RCT Follow up: providers: DONE patients: NOT CLEAR Blinded assessment: NOT CLEAR Baseline: DONE Reliable outcomes: DONE Protection against contamination: DONE Overall quality: MODERATE
Participants	26 primary care physicians at nine clinics Country: USA Proportion of eligible providers who participated: NOT CLEAR General practice Academic/Teaching setting: NOT CLEAR Type of targeted behaviour: PREVENTIVE CARE (teaching sigmoidoscopy in prevention of cancer) Complexity of targeted behaviour: MEDIUM
Interventions	1. CME: two afternoons with didactic presentation, educational material, discussions and skills training (7 sigmoidoscopies) 2. No intervention control at first, then same as above
Outcomes	Professional practice: rate of sigmoidoscopy per 1000 patients before and after sigmoidoscopy skills preceptorship for both groups Patient: none Seriousness of outcome: MODERATE
Notes	Not analysed as a randomised controlled trial ‐ data not extractable
Risk of bias
Bias	Authors' judgement	Support for judgement
Allocation concealment (selection bias)	Unclear risk	B ‐ Unclear

Roter 1995.

Methods	Cluster RCT Follow up: providers: NOT DONE patients: DONE Blinded assessment: DONE Baseline: NOT DONE Reliable outcomes: DONE Protection against contamination: NOT CLEAR Overall quality: LOW
Participants	88 primary care physicians and 648 of their patients Country: USA Proportion of eligible providers who participated: 16% General practice Academic/Teaching setting: NOT CLEAR Type of targeted behaviour: GENERAL MANAGEMENT OF A PROBLEM (physicians' communication skills for detection of emotional distress) Complexity of targeted behaviour: MEDIUM
Interventions	1. CME: 4‐hr session x 2 focusing on emotion‐handling skills, one week apart with homework in between: tape recording of one patient to be discussed at the second session 2. 4‐hr session x 2 focusing on problem defining skills, one week apart with homework in between: tape recording of one patient to be discussed at the second session 3. No intervention control
Outcomes	Professional practice: number of emotional and problem‐defining responses of physicians in patients visits Patient: patients' distress scores at 6 months Seriousness of outcome: MODERATE
Notes
Risk of bias
Bias	Authors' judgement	Support for judgement
Allocation concealment (selection bias)	Unclear risk	B ‐ Unclear

Rowlands 2003.

Methods	Cluster RCT Follow up: providers: DONE patients: N/A Blinded assessment: DONE Baseline: NOT CLEAR Reliable outcomes: DONE Protection against contamination: DONE Overall quality: MODERATE
Participants	26 practices with four or more partners Country: UK Proportion of eligible providers who participated: 9% General practice Academic/Teaching setting: NOT CLEAR Type of targeted behaviour: other (referrals to specialist services) Complexity of targeted behaviour: LOW
Interventions	1. CME: several within‐practice educational meetings, total of 5 hrs on average for each practice 2. No intervention control
Outcomes	Professional practice: referral rates Patient: none Seriousness of outcome: LOW
Notes
Risk of bias
Bias	Authors' judgement	Support for judgement
Allocation concealment (selection bias)	Low risk	A ‐ Adequate

Santoso 1996.

Methods	Cluster RCT Follow up: providers: NOT CLEAR patients: N/A Blinded assessment: NOT CLEAR Baseline: DONE Reliable outcomes: DONE Protection against contamination: DONE Overall quality: MODERATE
Participants	6 districts with 15 health centers randomly selected from each district. Country: Indonesia Proportion of eligible providers who participated: 100% Community‐based care Academic/Teaching setting: NOT CLEAR Type of targeted behaviour: Prescribing (appropriate use of drugs) Complexity of targeted behaviour: LOW
Interventions	1. CME: 2‐hr interactive small group face‐to‐face intervention at the health centre, 8‐12 participants 2. CME: 2‐hr didactic seminar with 60‐80 participants 3. No intervention control
Outcomes	Professional practice: % increase in use of oral rehydration therapy Patient: none Seriousness of outcome: HIGH
Notes
Risk of bias
Bias	Authors' judgement	Support for judgement
Allocation concealment (selection bias)	Low risk	A ‐ Adequate

Schectman 2003.

Methods	Cluster RCT Follow up: providers: DONE patients: N/A Blinded assessment: DONE Baseline: DONE Reliable outcomes: DONE Protection against contamination: DONE Overall quality: HIGH
Participants	14 groups with 120 primary care physician and associate practitioners from 2 group model HMO practices Country: USA Proportion of eligible providers who participated: NOT CLEAR General practice Academic/Teaching setting: NOT CLEAR Type of targeted behaviour: GENERAL MANAGEMENT OF A PROBLEM (guideline consistent behaviour for provision of services for low back pain) Complexity of targeted behaviour: MEDIUM
Interventions	1. CME: 1.5 hrs physician education + audit & feedback + follow‐up visit 2. Patient education materials (videotape and pamphlet) + one visit from the study investigators + 2 written reminders 3. Both interventions 4. No intervention control
Outcomes	Professional practice: % utilization of at least 1 of 4 indicators of guideline consistent behaviour (on basis of patient care episodes) Patient: none Seriousness of outcome: MODERATE
Notes
Risk of bias
Bias	Authors' judgement	Support for judgement
Allocation concealment (selection bias)	Low risk	A ‐ Adequate

Scheel 2002.

Methods	Cluster RCT Follow up: providers: DONE patients: DONE Blinded assessment: DONE Baseline: NOT DONE? Reliable outcomes: DONE Protection against contamination: DONE Overall quality: HIGH
Participants	General practitioners, back pain patients, their employers and local National Insurance Administration staff in 65 municipalities in 3 counties Country: Norway Proportion of eligible providers who participated: 100% Primary care Academic/Teaching setting: NON‐TEACHING Type of targeted behaviour: GENERAL MANAGEMENT OF A PROBLEM (use of active sick leave for back pain) Complexity of targeted behaviour: MEDIUM
Interventions	1. CME: continuing education workshop for GPs on low back pain and active sick leave + targeted information to patients, the local National Insurance Administration staff and employers + a new check box in the form for reporting sick leave (=reminder) + a standard agreement plan between employer and employee for rehabilitation + desktop summary for GPs of clinical practice guidelines + resource person for each region to support GPs and follow‐up patients on sick leave for >16 days 2. No intervention control + a third group having a passive strategy: targeted information, check box in report of sick leave, standard agreement and desktop summary
Outcomes	Professional practice: % of patients on active sick leave Patient: none Seriousness of outcome: LOW
Notes	No baseline data
Risk of bias
Bias	Authors' judgement	Support for judgement
Allocation concealment (selection bias)	Low risk	A ‐ Adequate

Simons 2001.

Methods	Cluster RCT Follow up: providers: NOT CLEAR patients: NOT DONE Blinded assessment: NOT CLEAR Baseline: NOT DONE Reliable outcomes: NOT CLEAR Protection against contamination: DONE Overall quality: MODERATE
Participants	Health visitors in 18 clinics Country: UK Proportion of eligible providers who participated: NOT CLEAR Community‐based care Academic/Teaching setting: NON‐TEACHING Type of targeted behaviour: PREVENTIVE CARE (help couples with relationship problems after birth) Complexity of targeted behaviour: MEDIUM
Interventions	1. CME: 4‐day training 2. No intervention control
Outcomes	Professional practice: % of mothers remembering having discussed relationship with health visitor Patient: none Seriousness of outcome: LOW
Notes
Risk of bias
Bias	Authors' judgement	Support for judgement
Allocation concealment (selection bias)	Low risk	A ‐ Adequate

Smeele 1999.

Methods	Cluster RCT Follow up: providers: DONE patients: NOT DONE Blinded assessment: NOT CLEAR Baseline: NOT DONE Reliable outcomes: NOT CLEAR Protection against contamination: DONE Overall quality: MODERATE
Participants	34 general practitioners with 433 asthma/COPD patients Country: Netherlands Proportion of eligible providers who participated: 63% General practice Academic/Teaching setting: NOT CLEAR Type of targeted behaviour: GENERAL MANAGEMENT OF A PROBLEM (improve care for asthma patients) Complexity of targeted behaviour: MEDIUM
Interventions	1. CME: 2hr x 4 interactive meetings with peer review included + one educational session for practice assistants 2. No intervention control
Outcomes	Professional practice: only self‐reported Patient: patients' self‐reported scores of quality of life Seriousness of outcome: HIGH
Notes
Risk of bias
Bias	Authors' judgement	Support for judgement
Allocation concealment (selection bias)	Unclear risk	B ‐ Unclear

Smith 1995.

Methods	Cluster RCT Follow up: providers: NOT DONE patients: N/A Blinded assessment: DONE Baseline: NOT DONE Reliable outcomes: NOT CLEAR Protection against contamination: NOT CLEAR Overall quality: LOW
Participants	87 obstetricians and midwives in five hospitals agreed to participate Country: UK Proportion of eligible providers who participated: 69%. 35 providers completed the study Hospital, inpatients Academic/Teaching setting: MIXED Type of targeted behaviour: SCREENING (communication of test results) Complexity of targeted behaviour: LOW
Interventions	1. CME: 1‐hour video‐based training session 2. 1‐hour video‐based training session + feedback on audio taped consultations 3. No intervention control
Outcomes	Professional practice: scores for information‐giving and communication skills Patient: none Seriousness of outcome: LOW
Notes
Risk of bias
Bias	Authors' judgement	Support for judgement
Allocation concealment (selection bias)	Unclear risk	B ‐ Unclear

Solomon 2004.

Methods	Provider RCT Follow up: providers: DONE patients: DONE Blinded assessment: NOT CLEAR Baseline: DONE Reliable outcomes: NOT CLEAR Protection against contamination: NOT CLEAR Overall quality: LOW
Participants	21 rheumatologists at 1 large academic arthritis practice Country: USA Proportion of eligible providers who participated: 100% Hospital, outpatients Academic/Teaching setting: university‐based Type of targeted behaviour: PRESCRIBING (glucocorticoid‐induced osteoporosis) Complexity of targeted behaviour: LOW
Interventions	1. CME: 1.5 hr educational dinner meeting, mixed format (feedback material discussed) + list of patients with rheumatoid arthritis sent to doctors (feedback) 3 weeks later + reminder guideline 2. No intervention control
Outcomes	Professional practice: % of patients having medication for osteoporosis Patient: none Seriousness of outcome: LOW
Notes
Risk of bias
Bias	Authors' judgement	Support for judgement
Allocation concealment (selection bias)	Unclear risk	B ‐ Unclear

Strecher 1991.

Methods	Cluster RCT Follow up: providers: DONE patients: NOT DONE Blinded assessment: NOT CLEAR Baseline: NOT DONE Reliable outcomes: NOT CLEAR Protection against contamination: NOT DONE Overall quality: LOW
Participants	261 residents in 11 primary care training programmes providing smoking counselling Country: USA Proportion of eligible providers who participated: 96% Hospital, outpatients Academic/Teaching setting: UNIVERSITY‐BASED Type of targeted behaviour: PREVENTIVE CARE (smoking counselling) Complexity of targeted behaviour: LOW
Interventions	1. CME: 1‐hour tutorial including 10‐min slide presentation, 10 min counselling approach, 20‐min group discussion + 1‐hour small group or individual follow up 2. Same tutorial + prompting program (chart‐based reminders) 3. Prompting program
Outcomes	Professional practice: counselling frequency, mean number of five techniques used per patient, 5 counselling techniques Patient: none Seriousness of outcome: MODERATE
Notes
Risk of bias
Bias	Authors' judgement	Support for judgement
Allocation concealment (selection bias)	Unclear risk	B ‐ Unclear

Sulmasy 1996.

Methods	Cluster RCT Follow up: providers: NOT CLEAR patients: NOT CLEAR Blinded assessment: NOT CLEAR Baseline: DONE Reliable outcomes: NOT DONE Protection against contamination: DONE Overall quality: LOW
Participants	88 medical residents and 250 of their patients Country: USA Proportion of eligible providers who participated: 100% General practice Academic/Teaching setting: UNIVERSITY‐BASED Type of targeted behaviour: the making of advance directives Complexity of targeted behaviour: LOW
Interventions	1. CME: 0.5‐hr pre clinic lecture on the topic of advance directives + videotape included + a videotaped session of their own patient interview with feedback afterwards=skills training 2. Control: NONE
Outcomes	Professional practice: % of charts documenting a discussion regarding advance directives Patient: none Seriousness of outcome: LOW
Notes
Risk of bias
Bias	Authors' judgement	Support for judgement
Allocation concealment (selection bias)	Unclear risk	B ‐ Unclear

Thom 1999.

Methods	Cluster RCT Follow up: providers: DONE patients: NOT CLEAR Blinded assessment: DONE Baseline: DONE Reliable outcomes: NOT CLEAR Protection against contamination: DONE Overall quality: MODERATE
Participants	20 family practice physicians Country: USA Proportion of eligible providers who participated: 31% Family practitioners Academic/Teaching setting: NOT CLEAR Type of targeted behaviour: OTHER (building patients' trust in their physician) Complexity of targeted behaviour: HIGH
Interventions	1. CME: problem‐based, small group discussions with brief didactic presentations, viewing of videotaped patient encounters and role‐playing, 7 hrs 2. Control: NONE
Outcomes	Professional practice: no primary specified ‐ no baseline so cannot choose the highest? Patient: mean trust score Seriousness of outcome: HIGH
Notes
Risk of bias
Bias	Authors' judgement	Support for judgement
Allocation concealment (selection bias)	Low risk	A ‐ Adequate

Thompson C 2000.

Methods	Cluster RCT Follow up: providers: DONE patients: DONE Blinded assessment: DONE Baseline: DONE (for patient data) Reliable outcomes: NOT CLEAR Protection against contamination: DONE Overall quality: HIGH
Participants	169 physicians in 59 primary care practices Country: UK Proportion of eligible providers who participated: 26% General practice Academic/Teaching setting: NON‐TEACHING Type of targeted behaviour: GENERAL MANAGEMENT OF A PROBLEM (detection and outcome of depression) Complexity of targeted behaviour: LOW
Interventions	1. CME: 4‐hr mixed format seminar: videos, role play, small‐group discussion cases + guideline + educators remained available to the practices for about 9 months 2. Control: NONE
Outcomes	Professional practice: % detection of depressive patients Patient: % of patients with HAD depression score >= 8 Seriousness of outcome: MODERATE
Notes
Risk of bias
Bias	Authors' judgement	Support for judgement
Allocation concealment (selection bias)	Low risk	A ‐ Adequate

Thompson RS 2000.

Methods	Cluster RCT Follow up: providers: DONE patients: NOT CLEAR Blinded assessment: DONE Baseline: DONE Reliable outcomes: DONE Protection against contamination: DONE Overall quality: MODERATE
Participants	208 providers (mixed) in 5 primary care clinics of a large health maintenance organization Country: USA Proportion of eligible providers who participated: NOT CLEAR General practice Academic/Teaching setting: NOT CLEAR Type of targeted behaviour: PREVENTIVE CARE (identification and management of domestic violence) Complexity of targeted behaviour: LOW
Interventions	1. CME: 2 half‐day training sessions + extra training for opinion leaders + bimonthly newsletter + 4 clinic educational rounds + system support: posters, cue cards, questionnaires + feedback of results 2. Control: NONE
Outcomes	Professional practice: % of patients being asked about domestic violence Patient: none Seriousness of outcome: HIGH
Notes
Risk of bias
Bias	Authors' judgement	Support for judgement
Allocation concealment (selection bias)	Unclear risk	B ‐ Unclear

Van der Weijden 1999.

Methods	Cluster RCT Follow up: providers: DONE patients: NOT CLEAR Blinded assessment: DONE Baseline: DONE Reliable outcomes: DONE Protection against contamination: DONE Overall quality: HIGH
Participants	32 general practitioners in 20 practices, 3950 patient records Country: Netherlands Proportion of eligible providers who participated: NOT CLEAR General practitioners Academic/Teaching setting: NON‐TEACHING Type of targeted behaviour: TEST ORDERING (performance in daily practice regarding targeted cholesterol testing) Complexity of targeted behaviour: LOW
Interventions	1. CME: interactive teaching by local opinion leaders in moderate group size: 3 hrs + consultation registration form (reminders) + desktop flow chart of guideline + patient education leaflet + 2 outreach visits/face‐to‐face instruction at workplace 2. Control: postal distribution of guideline with scientific background materials
Outcomes	Professional practice: median proportion of patients for whom the GP performed repeat testing to diagnose hypercholesterolaemia Patient: none Seriousness of outcome: LOW
Notes
Risk of bias
Bias	Authors' judgement	Support for judgement
Allocation concealment (selection bias)	Low risk	A ‐ Adequate

Varroud‐Vial 2004.

Methods	Cluster RCT Follow up: providers: DONE patients: NOT DONE Blinded assessment: DONE Baseline: DONE Reliable outcomes: DONE Protection against contamination: DONE Overall quality: MODERATE
Participants	67 general practitioners in four separate districts of one region Country: France Proportion of eligible providers who participated: NOT CLEAR Family practitioners Academic/Teaching setting: NON_TEACHING Type of targeted behaviour: GENERAL MANAGEMENT OF A PROBLEM (glycaemic control of patients with Type 2 diabetes in a primary care setting) Complexity of targeted behaviour: LOW
Interventions	1. CME, staged diabetes management program: 3.5 hrs x 3 educational sessions 2. Control: no intervention control
Outcomes	Professional practice: indicators for treatment of patients Patient: HbA1c Seriousness of outcome: MODERATE
Notes
Risk of bias
Bias	Authors' judgement	Support for judgement
Allocation concealment (selection bias)	Unclear risk	B ‐ Unclear

Veninga 1999.

Methods	Cluster RCT Follow up: providers: NOT CLEAR patients: N/A Blinded assessment: DONE Baseline: DONE Reliable outcomes: DONE Protection against contamination: DONE Overall quality: MODERATE
Participants	665 physicians from general practice Country: Netherlands, Sweden, Slovakia and Norway Proportion of eligible providers who participated: NL 24%; S 35%; SK 20%; N 31% General practice Academic/Teaching setting: NOT CLEAR Type of targeted behaviour: PRESCRIBING (asthma care) Complexity of targeted behaviour: LOW
Interventions	1. CME: 2 educational meetings (self‐learning method in small peer groups) on asthma care + individual feedback presented in group for discussion 2. 2 educational meetings (self‐learning method in small peer groups) on the care of urinary tract infection + individual feedback presented in group for discussion
Outcomes	Professional practice: % correct prescribing for asthma Patient: none Seriousness of outcome: LOW
Notes
Risk of bias
Bias	Authors' judgement	Support for judgement
Allocation concealment (selection bias)	Low risk	A ‐ Adequate

Verstappen 2003.

Methods	Cluster RCT Follow up: providers: DONE patients: N/A Blinded assessment: DONE Baseline: DONE Reliable outcomes: DONE Protection against contamination: DONE Overall quality: HIGH
Participants	174 primary care physicians in 26 local primary care practice groups in 5 health care regions Country: Netherlands Proportion of eligible providers who participated: 70% Primary care Academic/Teaching setting: NON‐TEACHING Type of targeted behaviour: TEST ORDERING (reducing inappropriate test ordering) Complexity of targeted behaviour: LOW
Interventions	1. CME: multifaceted with interactive educational small group meeting + audit and feedback (moderate) for 3 selected clinical problems 2. Control: multifaceted with interactive educational small group meeting + audit and feedback (moderate) for 3 other selected clinical problems
Outcomes	Professional practice: mean number of inappropriate tests, per physician per 6 months Patient: none Seriousness of outcome: LOW
Notes
Risk of bias
Bias	Authors' judgement	Support for judgement
Allocation concealment (selection bias)	Low risk	A ‐ Adequate

Verstappen 2004.

Methods	Cluster RCT Follow up: providers: DONE patients: N/A Blinded assessment: DONE Baseline: DONE Reliable outcomes: DONE Protection against contamination: DONE Overall quality: HIGH
Participants	194 primary care physicians in 27 local primary care practice groups in 5 health care regions Country: Netherlands Proportion of eligible providers who participated: 71% Primary care Academic/Teaching setting: NON‐TEACHING Type of targeted behaviour: TEST ORDERING (reducing inappropriate test ordering) Complexity of targeted behaviour: LOW
Interventions	1. CME: multifaceted with interactive educational small group meeting + audit and feedback (moderate) 2. Control: audit and feedback
Outcomes	Professional practice: mean number of tests per physician Patient: none Seriousness of outcome: LOW
Notes
Risk of bias
Bias	Authors' judgement	Support for judgement
Allocation concealment (selection bias)	Low risk	A ‐ Adequate

Ward 1996.

Methods	Cluster RCT Follow up: providers: DONE patients: DONE Blinded assessment: DONE Baseline: DONE Reliable outcomes: DONE Protection against contamination: DONE Overall quality: MODERATE
Participants	34 general practice trainees providing preventive care for 1500 patients Country: Australia Proportion of eligible providers who participated: 50% General practice Academic/Teaching setting: UNIVERSITY‐BASED Type of targeted behaviour: PREVENTIVE CARE (stop‐smoking counselling) Complexity of targeted behaviour: MODERATE
Interventions	1. CME, mixed: 3‐day workshop (didactic presentation, small group skill practice, role‐play) 2. Control: 3‐day workshop in rational prescribing
Outcomes	Professional practice: number of patients asked about smoking status Patient: none Seriousness of outcome: MODERATE
Notes
Risk of bias
Bias	Authors' judgement	Support for judgement
Allocation concealment (selection bias)	Low risk	A ‐ Adequate

Welschen 2004.

Methods	Cluster RCT Follow up: providers: DONE patients: N/A Blinded assessment: DONE Baseline: DONE Reliable outcomes: DONE Protection against contamination: DONE Overall quality: HIGH
Participants	12 peer review groups including 100 general practitioners with their collaborating pharmacists Country: Netherlands Proportion of eligible providers who participated: 29% General practice Academic/Teaching setting: NOT CLEAR Type of targeted behaviour: PRESCRIBING (of antibiotics for respiratory tract symptoms) Complexity of targeted behaviour: LOW
Interventions	1. CME: peer‐group education meeting with communication skills training + feedback presented at practice level + 2‐hr group education for assistants + education material for patients at practice site 2. Control: No intervention control
Outcomes	Professional practice: rate of antibiotic prescribing in % Patient: patient satisfaction Seriousness of outcome: LOW
Notes
Risk of bias
Bias	Authors' judgement	Support for judgement
Allocation concealment (selection bias)	Low risk	A ‐ Adequate

Westphal 1995.

Methods	Cluster RCT Follow up: providers: DONE patients: N/A Blinded assessment: NOT CLEAR Baseline: DONE Reliable outcomes: NOT CLEAR Protection against contamination: DONE Overall quality: MODERATE
Participants	Health professionals in 8 maternity hospitals providing advice about breast feeding Country: Brazil Proportion of eligible providers who participated: NOT CLEAR Hospital setting Academic/Teaching setting: University based/teaching setting Type of targeted behaviour: PREVENTIVE CARE (breast‐feeding practice) Complexity of targeted behaviour: HIGH
Interventions	1. CME: mixed format full‐time for 14 days over 3 weeks 2. Control: no intervention control
Outcomes	Professional practice: score (1‐10) for institutional change according to WHO's 10 steps for successful breast‐feeding Patient: none Seriousness of outcome: LOW
Notes
Risk of bias
Bias	Authors' judgement	Support for judgement
Allocation concealment (selection bias)	Unclear risk	B ‐ Unclear

White 1985.

Methods	Cluster RCT Follow up: providers: DONE patients: NOT CLEAR Blinded assessment: DONE Baseline: DONE Reliable outcomes: NOT DONE Protection against contamination: DONE Overall quality: MODERATE
Participants	103 family doctors or general internists in 12 communities caring for in‐patients post‐myocardial infarction Country: USA Proportion of eligible providers who participated: 71% Family practitioners Academic/Teaching setting: NOT CLEAR Type of targeted behaviour: GENERAL MANAGEMENT OF A PROBLEM (care for acute myocardial infarction) Complexity of targeted behaviour: MEDIUM
Interventions	1. CME: 3.5‐hr educational session: 2 hrs with traditional methods and 1.5 h of discussions and case examples 2. Control: no intervention control
Outcomes	Professional practice: overall measures of desired patient care Patient: none Seriousness of outcome: HIGH
Notes
Risk of bias
Bias	Authors' judgement	Support for judgement
Allocation concealment (selection bias)	Unclear risk	B ‐ Unclear

Wilson 1992.

Methods	Cluster RCT Follow up: providers: DONE patients: DONE Blinded assessment: DONE Baseline: DONE Reliable outcomes: NOT CLEAR Protection against contamination: DONE Overall quality: MODERATE
Participants	22 family doctors providing exercise counselling for 420 patients Country: CANADA Proportion of eligible providers who participated: 12% General practice Academic/Teaching setting: NON‐TEACHING Type of targeted behaviour: PREVENTIVE CARE (exercise counselling) Complexity of targeted behaviour: LOW
Interventions	1. CME: 2‐hour workshop with three components: discussion forum, practical teaching and overview of resources + mailed material 2. Control: no intervention control
Outcomes	Professional practice: % of patients with whom physicians discussed exercise, as reported by patients Patient: none Seriousness of outcome: MODERATE
Notes
Risk of bias
Bias	Authors' judgement	Support for judgement
Allocation concealment (selection bias)	Low risk	A ‐ Adequate

Wirtschafter 1986.

Methods	Cluster RCT Follow up: providers: NOT CLEAR patients: NOT CLEAR Blinded assessment: NOT CLEAR Baseline: NOT DONE Reliable outcomes: NOT DONE Protection against contamination: DONE Overall quality: LOW
Participants	Estimated 1097 doctors (family practitioners + paediatricians) and nurses in 40 community hospitals caring for newborns in neonatal intensive care units Country: USA Proportion of eligible providers who participated: NOT CLEAR Hospital Academic/Teaching setting: NON‐TEACHING Type of targeted behaviour: GENERAL MANAGEMENT OF A PROBLEM (respiratory distress in neonatals) Complexity of targeted behaviour: HIGH
Interventions	1. CME: 1.5 hrs x 2 given 6‐8 months apart: mediated lecture/case study presentation + monthly newsletter 2. 1.5 hrs x 2 given 6‐8 months apart: mediated lecture/case study presentation + protocol for treatment of respiratory distress + newsletter 3. Control: newsletter
Outcomes	Professional practice: combined process score Patient: neonatal mortality Seriousness of outcome: HIGH
Notes
Risk of bias
Bias	Authors' judgement	Support for judgement
Allocation concealment (selection bias)	Unclear risk	B ‐ Unclear

Woodcock 1999.

Methods	Cluster RCT Follow up: providers: NOT CLEAR patients: NOT DONE Blinded assessment: DONE Baseline: NOT DONE Reliable outcomes: NOT CLEAR Protection against contamination: DONE Overall quality: LOW
Participants	107 general practitioners and nurses in 41 practices Country: UK Proportion of eligible providers who participated: NOT CLEAR General practice Academic/Teaching setting: NON‐TEACHING Type of targeted behaviour: GENERAL MANAGEMENT OF A PROBLEM (patient centred diabetes care) Complexity of targeted behaviour: LOW
Interventions	1. CME: general practitioners: 0.5‐day group training; nurses: 1.5 days 2. Control: no intervention control
Outcomes	Professional practice: patients' recognition of a patient booklet, recognition of an insert to the booklet, patients' reports of nurse consulting behaviour Patient: none Seriousness of outcome: MODERATE
Notes	No baseline data
Risk of bias
Bias	Authors' judgement	Support for judgement
Allocation concealment (selection bias)	Low risk	A ‐ Adequate

Worrall 1999.

Methods	Cluster RCT Follow up: providers: DONE patients: NOT CLEAR Blinded assessment: NOT CLEAR Baseline: DONE Reliable outcomes: NOT CLEAR Protection against contamination: DONE Overall quality: MODERATE
Participants	42 family physicians Country: Canada Proportion of eligible providers who participated: 41% General practice Academic/Teaching setting: NOT CLEAR Type of targeted behaviour: GENERAL MANAGEMENT OF A PROBLEM (management of depression) Complexity of targeted behaviour: MEDIUM
Interventions	1. CME: 3‐hr small group educational session: teaching and case‐based discussion + psychiatrist help service available once a week + clinical guidelines 2. Control: mailed clinical guidelines on the management of depression
Outcomes	Professional practice: % of correct diagnoses of depression Patient: mean patient score on the Centre for Clinical Epidemiological Studies Depression scale Seriousness of outcome: MODERATE
Notes
Risk of bias
Bias	Authors' judgement	Support for judgement
Allocation concealment (selection bias)	Low risk	A ‐ Adequate

Characteristics of excluded studies [ordered by study ID]

Study	Reason for exclusion
Camp‐Sorrell 1991	Control group received education
Campbell 1991	Outcomes were not measured in a clinical situation
Carlsson 1998	Outcomes were not measured in a clinical situation
Casebeer 1999	Audioconferences
Davidoff 1989	Control group received education
Doyne 2004	Educational outreach
Dunn 1992	Outcomes were not measured in a clinical situation
Eckstrom 1999	Quasirandomised
Francke 1997	Outcomes were based on self‐report
Gifford 1999	Outcomes measured in a test situation
Huang 2002	Unclear whether the outcomes were reported by participants or whether they were observed by others
Langewitz 1998	Outcomes were not measured in a clinical situation
Lundgren 1999	Not described as educational meeting
Martin 2004	Testing of an algorithm for improving nutritional support, not of an educational intervention
Morrison 2001 1	Testing the effectiveness of guidelines, not an educational meeting
O'Neill 1999	Not described as educational meeting
Ockene1995	Outcomes were not measured in a clinical situation
Pinkerton 1980	The intervention was video‐watching
Premaratne 1999	Testing of the effectiveness of an asthma centre, not of an educational meeting
Proctor 1999	Educational outreach
Quirk 1991	Outcomes were not measured in a clinical situation
Ratanajamit 2002	The participants were drugstore personnel
Ravaud 2004	We defined the intervention as educational outreach
Roter 1990	Outcomes were not measured in a clinical situation
Ruiz Moral 2001	Outcomes were measured by patients that the physicians knew were fake (test situation)
Saturno 1995	Outcomes were based on self‐report
Sibley 1982	The intervention was printed material, not educational meeting
Stross 1983	Outcomes were not measured in a clinical situation
Sulmasy 1992	The participants were physicians under education
Terry 1981	Outcomes were not measured in a clinical situation
Tziraki 2000	The outcome measures (adherence scores) were a mixture of subjective and objective measures.
Wedge 2005	Not described as educational meeting
Zwar 1995	Outcomes were based on self‐report

Differences between protocol and review

None

Contributions of authors

Task	Contributor
Drafted the protocol	LF, ADO, GJ, AB
Searched for trials	EPOC
Scanned titles and abstracts for eligibility	LF, AB
Obtained copies of potentially eligible trials	LF
Appraised and select which trials to include	LF/AB; LF/AR; LF/GJ; LF/MAO; LF/FW; LF/DAD
Extracted data from trials	LF/AB; LF/AR; LF/GJ; LF/MAO; LF/FW; LF/DAD
Entered data	LF
Carried out the analysis	JOJ
Interpreted the analysis	All authors
Drafted the final review	LF, ADO
Prepared tables and figures	JOJ, ADO, LF
All authors reviewed the draft of the protocol and the systematic review

 

Declarations of interest

None

Edited (no change to conclusions)