Professional interventions to implement guidelines to prevent hazardous alcohol consumption by patients in primary care settings

Abstract

This is a protocol for a Cochrane Review (Intervention). The objectives are as follows:

To determine the effectiveness of strategies targeting primary healthcare professionals to implement clinical guidelines on the prevention of hazardous or harmful alcohol consumption in patients attended to in primary care settings. We will evaluate the effects on processes and outcomes of care that are attributable to the implementation or dissemination strategies.

Background

Description of the condition

Excessive consumption of alcoholic beverages can be defined in relation to the amount of alcohol consumed and to the physical and psychological consequences associated with its consumption. According to these criteria, there are three categories related to excessive consumption of alcohol: hazardous drinking, harmful drinking, and alcohol dependence (SIGN 2003).

The excessive consumption of alcohol represents a public health problem that has a strong impact on physical and psychological health. Moreover, social harm from drinking can be classified in terms of how it affects important roles and responsibilities of everyday life: work, family, friendship, and public character (WHO 2011). Alcohol is implicated as a risk factor in over 60 health disorders: acute, such as stupor, vomiting and respiratory distress; as well as chronic problems, such as dependence, depression or cirrhosis of the liver (WHO 2012). The greater part of the damage and social and health costs associated with alcohol are produced by non‐dependent consumers (Funk 2005). In fact, even moderate consumption of alcohol causes a risk to the drinker, especially associated with an increase in accidents (NCIPC 2006). Alcohol consumption is the world’s third largest risk factor for disease and disability; in middle‐income countries it is the greatest risk (WHO 2011). In Europe it is the third cause of premature death, after smoking and hypertension (Anderson 2006). Yet, despite all these problems, the harmful use of alcohol remains a low priority in public policies and many lesser health risks have higher priority.

In the primary care setting there are effective interventions available to prevent the excessive consumption of alcohol, such as providing brief interventions (BI) for those patients screened for hazardous drinking and determined to be at risk (Agosti 1995; Beich 2003; Ballesteros 2004; Whitlock 2004; Bertholet 2005; Kaner 2007; Kaner 2009). Screening is based on the application of the Alcohol Use Disorders Identification Test (AUDIT) (Babor 2001) to the population at risk for excessive alcohol consumption, or to individuals who require it. BI, which are directed to non‐dependent drinkers, aim to help someone reduce their alcohol consumption (sometimes even to abstain); they can comprise either a short session of structured brief advice or a longer, more motivationally‐based session (that is, an extended brief intervention); and both can be carried out by non‐alcohol specialists (NICE 2010).The efficacy of BI is well documented in numerous meta‐analyses (Beich 2003; Ballesteros 2004; Whitlock 2004; Kaner 2007). A Cochrane systematic review (Kaner 2007) concluded that BI, delivered in general practice or primary care to patients presenting to primary care not specifically for alcohol treatment, consistently produced reductions in alcohol consumption.

However, the compliance of primary care physicians with these interventions is rather low. Only 34% of physicians do a screening test for excessive alcohol consumption during their patients’ annual visits (Spandorfer 1999) and less than half of patients with excessive alcohol consumption are identified (Kaner 2001; Berner 2007). There is, therefore, vast room for improvement in the prevention of excessive alcohol consumption in the primary care setting (Kaner 2001; Town 2006; Berner 2007). As a result of this, many patients have a condition that is both preventable and treatable.

Description of the intervention

There is consensus on the need to incorporate measures for the prevention of excessive alcohol consumption within the practice of primary care physicians. This has been stated in several guidelines, protocols and recommendations (SIGN 2003; Anderson 2008). However, the existence of guidelines and recommendations is not enough to promote changes in clinical practice (Farquhar 2002). In addition, a planned strategy for dissemination and implementation is necessary (Kryworuchko 2009).

There are effective strategies to promote the adherence of physicians to recommended clinical practice (Grimshaw 2004). These strategies can be directed to different levels: 1) physicians and their patients; 2) health centres and health systems; and 3) communities and the general population (Babor 2000). Additionally, the Cochrane Effective Practice and Organisation of Care (EPOC) Group defines four strategies: 1) professional interventions (such as distribution of educational materials or educational meetings); 2) financial interventions (such as fee‐for‐service or prospective payment); 3) organisational interventions (such as creation of clinical multidisciplinary teams); or 4) regulatory interventions (any intervention that aims to change health services delivery or costs by regulation or law) (EPOC Data Collection Checklist 2009).

For this review, we will focus on strategies targeting professionals working in primary health care for the implementation or dissemination, or both, of guidelines or recommendations on hazardous or harmful alcohol consumption in patients attended to in this setting. Examples of these strategies are the following: distribution of educational materials, educational meetings, local consensus processes, educational outreach visits, local opinion leaders, patient mediated interventions, audit and feedback, reminders, marketing, or mass media interventions (Anderson 2004;Nilsen 2006; EPOC Data Collection Checklist 2009).

The effectiveness of each strategy depends on various factors, among them the type of change to be made, the place where the change is to be implemented, and the presence of barriers and facilitators (Grupo de trabajo sobre implementación de GPC 2009). However, there is scarce evidence on what strategies are most effective for each situation (for example, depending on the health sector or health problem that must be acted upon), and on how the effectiveness of the strategies varies in relation to the barriers and co‐existent factors (Grimshaw 2004; Grimshaw 2006).

How the intervention might work

The majority of people who consume excessive amounts of alcohol can be identified within the context of primary care, where there are effective interventions to prevent it and minimize the associated damages (USPTF 2004). In the last few years clinical trials and systematic reviews have been published on the effectiveness of strategies directed to professionals, such as educational outreach visits, that favour the implementation in primary care of recommendations for the prevention of excessive consumption of alcohol (Kaner 1999; Anderson 2004; Babor 2005; Funk 2005; Nilsen 2006; Anderson 2009). These strategies may be effective and could have a great impact on the population given the large number of patients seen in this health sector. Moreover, the implementation of strategies directed to professionals is feasible in many contexts.

Why it is important to do this review

At present there are effective strategies to improve the compliance of health professionals with existing recommendations. There are clinical trials and systematic reviews on the effectiveness of different strategies that promote the use by professionals in primary care of effective interventions to prevent excessive consumption of alcohol (Anderson 2004; Nilsen 2006). However, these systematic reviews require updating. Moreover, none of them focused on strategies directed to healthcare professionals and they did not attempt to determine which specific strategies are more effective in each situation (Grimshaw 2004; Grimshaw 2006). They also did not follow the current methodology of The Cochrane Collaboration (for example, none of these reviews used the risk of bias tool (Higgins 2011a)).

For these reasons, in order to define which are the most effective strategies to implement guidelines to prevent hazardous or harmful alcohol consumption by patients in primary care settings, a Cochrane systematic review should be done. This systematic review will focus on the effectiveness of strategies fulfilling these characteristics and targeting primary healthcare professionals.

Objectives

To determine the effectiveness of strategies targeting primary healthcare professionals to implement clinical guidelines on the prevention of hazardous or harmful alcohol consumption in patients attended to in primary care settings. We will evaluate the effects on processes and outcomes of care that are attributable to the implementation or dissemination strategies.

Methods

Criteria for considering studies for this review

Types of studies

This review will include randomised controlled trials (RCT) (EPOC Data Collection Checklist 2009), that is, experimental studies in which the participants (or groups of participants in cluster‐RCTs) are allocated to different interventions using methods that ensure random assignment (Norwegian Satellite EPOC Group). We will include cluster‐RCTs with at least two intervention sites and two control sites.

The minimum methodological inclusion criteria for RCTs (EPOC Data Collection Checklist 2009) will be:

• the objective measurement of performance or provider behaviour on health or patient outcome(s) in a clinical non‐test situation; and

• relevant and interpretable data presented or obtainable. 

If we are unsure of the study design, we will discuss the paper with the contact editor of theCochrane Effective Practice and Organisation of Care (EPOC) Group before data extraction is undertaken. We will include published and unpublished studies without any date or language restrictions.

Types of participants

Any study in which the majority of participants (> 50%) consisted of healthcare professionals (including physicians, nurses, psychologists, doctors assistants, social workers, and receptionists) working in primary care. We will consider an operative definition for primary care which will include all immediately accessible, general healthcare facilities which cover a broad range of possible presenting problems and which can be accessed by wide range of patients on demand, and not as the result of a referral for specialist care (Kaner 2007).

Types of interventions

Intervention

Any strategy targeting professionals working in primary health care for the implementation or dissemination, or both, of guidelines or recommendations on hazardous or harmful alcohol consumption in patients attended to in primary care settings. The combination of two or more strategies will also be considered as a valid intervention. We will consider any of the strategies detailed in section 2.1.1, "Professional interventions" (EPOC Data Collection Checklist 2009):

• distribution of educational materials;

• educational meetings;

• local consensus processes;

• educational outreach visits;

• local opinion leaders;

• patient mediated interventions;

• audit and feedback;

• reminders;

• marketing;

• mass media; and

• other categories to be agreed in consultation with the EPOC editorial team.

Comparative group

The comparative group should consist of one of the following elements:

• no intervention;

• standard practice; or

• any other strategy for the implementation or dissemination, or both, of guidelines or recommendations on hazardous or harmful alcohol consumption in patients attended to in primary care settings.

For this review, we will consider a broad definition for ‘clinical guideline’ or recommendation, for example: “systematically developed statements to assist practitioner decisions about appropriate health care for specific clinical circumstances” (IOM 1992). We will not require the term ‘guideline’ to be explicitly mentioned in the original study.

The interventions will include those that are exclusively focused on ‘hazardous’ or ‘harmful’ drinking, as well as alcohol related interventions that are part of broader prevention and health promotion activities. We will consider a broad definition for ‘hazardous’ or ‘harmful’ drinking: 'harmful drinking' will be defined as a pattern of drinking that is already causing damage to health. The damage may be either physical (for example, liver damage from chronic drinking) or mental (for example, depressive episodes secondary to drinking) (Babor 2001). Hazardous drinking will be defined as the regular consumption of alcohol that is associated with increased risk of harm. These consequences may be damage to health (physical or mental) or they may include social consequences to the drinker or others (Babor 2001; SIGN 2003). However, other definitions will also be accepted.

We will not consider as eligible those studies focusing only on dissemination or implementation strategies of guidelines on alcohol dependence. However, if identified, we will report these studies in the table 'Characteristics of excluded studies'.

Types of outcome measures

We will evaluate the effects on processes and outcomes of care attributable to the dissemination or implementation strategies. We will categorise eligible outcomes as ‘process of care outcomes’ and ‘patient clinical outcomes’.

Primary outcomes

A) Process of care outcomes

We will consider objective measures of provider compliance with guidelines or recommendations for hazardous or harmful drinking (for example, screening, measurement of alcohol intake, counselling, or making a follow‐up and referral). Process of care outcomes will be presented as:

• dichotomous data (e.g., the percentage of patients whose care was in compliance with a recommendation, or the percentage of patients whose drinking status was assessed); or

• continuous data, i.e., any continuous measure of how providers delivered care (e.g., number of patients referred to a specialist over three months, or time to detect a hazardous drinker).

When possible, we will present process of care outcomes as dichotomous data because dichotomous data are reported more frequently and because continuous process of care measures are less stable (Grimshaw 2004). We will not consider as eligible studies that only measure satisfaction or knowledge of professionals, or studies that only use aggregate self‐reported data on provider behaviour, for example, ‘do you usually screen for hazardous drinking during visits?’.

Direction of improvements in process of care outcomes: we will standardise the direction of effects so that increases will always correspond to improvements in care. For instance, if a study reports the proportion of patients who were not screened for hazardous drinking, we will record the complementary proportion of patients who received appropriate care. For this reason, a positive difference between post‐intervention percentages or means will be a desirable outcome.

B) Patient clinical outcomes

We will consider the following measures of drinking quantity, frequency, and intensity.

• Continuous data:

• • self‐ or other‐reports of drinking quantity, average consumption of alcohol per specified time period (grams/week);

  • self‐ or other‐reports of drinking frequency, number of drinking sessions per week;

  • self‐ or other‐reports of drinking intensity, number of drinks per drinking day (grams/drinking day);

  • physiological measures of alcohol consumption, e.g.e, serum gamma‐glutamyltransferase (GGT), or mean corpuscular volume (MCV).

• Dichotomous data:

  • proportion of participants recorded as a heavy drinker (someone who regularly exceeds the recommended limit for alcohol consumption). We will accept any definition supported by the study authors; and

  • proportion of participants recorded as a binge drinker (someone who drinks heavily in a short space of time to get drunk or feel the effects of alcohol). We will admit any definition supported by the study authors.

Secondary outcomes

Economic variables

Where information is available, relevant data on the cost effectiveness and resource utilisation will be reported and summarised in separated tables, but not synthesised. We will exclude those studies reporting only costs or other measures of resource use.

Search methods for identification of studies

The OVID Medline strategy (Appendix 1) has been developed by M. Fiander, EPOC Trials Search Co‐ordinator, in consultation with the authors and will be translated for other databases for the review. The finalised Medline strategy was developed by testing a variety of search terms and concept combinations. Results from test searches were provided to the primary authors who screened and marked relevant or potentially relevant citations. These citations were then used to validate and finalize the Medline strategy presented in this protocol.

The strategy combines the concepts of alcoholism/alcohol abuse with primary care or guideline terms. This comprehensive approach supports the broad definition of guideline provide in Types of interventions (above), and also agrees with EPOC Group search methodology which advocates broad topic searches due to the difficulty of identifying consistent language in the literature for professional and organisational interventions.

Search results will be restricted using the Cochrane Highly Sensitive Search Strategy for identifying randomised trials in MEDLINE: sensitivity‐ and precision‐maximizing version (2008 revision); and additional terms: multicenter study (publication type), random$ or cluster in title and abstract. Neither date nor language limits will be applied.

Primary studies will be identified using the following bibliographic databases, sources, and methods. Related systematic reviews will be identified by searching the Cochrane Database of Systematic Reviews, the Database of Abstracts of Reviews of Effectiveness (DARE), and the databases listed below.

Databases

MEDLINE, OVID (1946‐, In‐Process and other non‐indexed citations)

EMBASE, OVID (1947‐)

The Cochrane Central Register of Controlled Trials (CENTRAL), Wiley

CINAHL (Cumulative Index to Nursing and Allied Health Literature), EbscoHost (1980‐)

Cochrane EPOC Group, Specialised Register, Reference Manager

Cochrane Drugs and Alcohol Group, Specialised Register

Dissertations and Theses, ProQuest [1861‐]

LILACS (Latin American and Caribbean Health Sciences) Database, Virtual Health Library (VHL)

IBECS (Índice Bibliográfico Español en Ciencias de la Salud) Database, Virtual Health Library (VHL)

Open Grey, System for Information on Grey Literature in Europe (www.opengrey.eu/)

Web of Science, Conference Proceedings Citation Index‐ Science, Web of Knowledge (1990‐)

Centre for Reviews and Dissemination databases (www.crd.york.ac.uk/crdweb/)

Grey Literature

The following sources will be searched for literature not indexed in the databases listed above:

Joanna Briggs Institute (www.joannabriggs.edu.au/Search.aspx)

Commonwealth Fund (www.commonwealthfund.org/)

Robert Graham Centre (www.graham‐center.org)

Canadian Health Services Research Foundation (CHSRF) (www.chsrf.ca/Home.aspx)

Primary Health Care Research Information Service (PHCRIS) (www.phcris.org.au/)

TRIP (Turning Research Into Practice) Database (www.tripdatabase.com/)

Institute of Alcohol Studies (www.ias.org.uk/)

National Institute on Alcohol Abuse and Alcoholism (www.niaaa.nih.gov/)

National Institute for Health and Clinical Excellence (NICE) (www.nice.org.uk/)

Pan American Health Organization (PAHO) (new.paho.org/)

World Health Organization (WHO) (www.who.int/en/)

AHRQ Annual Conference (www.ahrq.gov/about/annlconf.htm)

Guidelines International Network (G‐I‐N) conference (http://www.g‐i‐n.net/)

World Organization of Family Doctors (WONCA) conference (www.globalfamilydoctor.com/conferences/conferences.asp?refurl=con)

International Society for Quality in Health Care (ISQua) International Conference (www.isqua.org/)

International Forum on Quality and Safety in Healthcare (http://internationalforum.bmj.com/)

Trial Registries & Trial Results

WHO International Clinical Trials Registry Platform (ICTRP) www.who.int/ictrp/en/

Clinical study results (www.clinicalstudyresults.org)

Additional search methods:

a)      Review reference lists of relevant systematic reviews or other publications.

b)      Contact authors of relevant studies or reviews to clarify reported published information or seek unpublished results/data.

c)      Contact researchers with expertise relevant to the review topic or EPOC interventions

d)      Conduct cited reference searches in ISI Web of Science/Web of Knowledge

Data collection and analysis

Selection of studies

Two review authors will screen each title and abstract looking for potentially relevant studies. Using a form developed to document the process (available upon request), we will classify the titles and abstracts into three groups: ‘exclude’, ‘potentially relevant’, or ‘unclear’. Any disagreement will be resolved through discussion. If finally there is no consensus, a third review author (Teresa Sanz Cuesta (TSC), Jesús López Alcalde (JLA), or Isabel del Cura (IDC)) will be consulted. At this stage, we will only exclude those papers classified by both review authors as ‘exclude’. All those titles classified as ‘unclear’ or ‘potentially relevant’ will be marked and their full text versions retrieved for definitive assessment of eligibility. We will try to obtain further information about studies published only as an abstract. If a full report is not available and the information cannot be obtained, we will exclude the abstract. If relevant, we will report its characteristics in the table 'Characteristics of excluded studies'.

Using a form developed to document the process, two review authors will classify the full text articles into three groups (‘exclude’, ‘unclear’, or ‘include’). Any disagreement will be resolved through discussion. If finally there is no consensus, a third review author (TSC, JLA, or IDC) or the editorial base of the Cochrane EPOC Group will be consulted. If relevant, we will document the disagreements. We will classify a study as ‘unclear’ when, due to any reason (such as insufficient information in the report), it is not clear if it fulfils the eligibility criteria of the review. If this is the case, we will ask the authors for clarification. If after two attempts we cannot obtain the information, we will finally exclude the study. We will detail in the table 'Characteristics of excluded studies' all the studies excluded that a user might reasonably expect to find in the review, with the reasons for their exclusion. The review authors will not be masked to the trial results, the publication details, or the institution during the selection of the studies.

We will include a study flow diagram, as recommended by the PRISMA statement (Lefebvre 2011).

Data extraction and management

Two review authors will independently perform the extraction of data from included studies (not being masked to the study details) using a data extraction form that was designed and pilot tested. This form (available upon request) will be based on the ‘EPOC data abstraction form’ (EPOC Data Abstraction Form 2007), the 'EPOC Data Collection Checklist' (EPOC Data Collection Checklist 2009), and the data collection templates used in similar systematic reviews (Grimshaw 2004); modified to capture more detailed information in some areas relevant to our review. We will abstract the following information from included studies: participants, setting and patients’ characteristics, methods, characteristics of intervention, control, co‐interventions (if present), outcomes, risk of bias, and results, among others. If there are several components in the intervention, we will create a graphical depiction of the experimental and control interventions using the PaT Plot tool (Perera 2007). Once created, a PaT Plot is easy to interpret and allows clear comparisons between different arms of a study (CEBM 2009).

Each pair of review authors will compare the extracted data and resolve disagreements by discussion until a consensus is reached. If there is no consensus, a third review author (TSC, JLA, or IDC) or the Cochrane EPOC Group will settle the discrepancies. We will document relevant disagreements and write to the study authors for any missing, incomplete, or unclear data. One review author (TSC) will enter the data in to Review Manager (RevMan 2011) and another one (Mercedes Rumayor Zarzuelo (MRZ)) will check the data entered manually. A third author (JLA) will check the whole process.

Reanalyses of study results

If a comparison is reanalysed, we will quote the P value associated with the effect estimate and annotate it with 'reanalysed’.

• Unit of analysis errors: if we detect 'unit of analysis errors' we will not attempt to reanalyse these data (see section Unit of analysis issues).

• Baseline imbalance: if the method of analysis of cluster‐RCTs did not account for eventual baseline differences of the outcomes, we will attempt to reanalyse the data.

• Within‐group comparisons: some of the comparisons may incorrectly compare results within groups instead of between the intervention and control groups. We will attempt to reanalyse the results in these comparisons. If the comparison concerns dichotomous data and the number of patients (or clusters) in the post‐intervention control and intervention groups can be abstracted, then we will reanalyse the comparison using a Chi² test. Similarly, if the outcome is continuous and both the standard deviations (SD) and the numbers of patients from both groups are known, then we will reanalyse the data of the trial with a t‐test using a pooled estimate of the standard deviation (Grimshaw 2004).

Dealing with duplicate publications

We will include only once those studies that have been published in duplicate, ensuring that all relevant data from all publications are extracted.

Assessment of risk of bias in included studies

Two review authors will independently assess the risk of bias (RoB) of each included study (not masked to the study details). They will resolve any disagreement by discussion and consensus, and by consulting a third review author (TSC, JLA or IDC) if necessary. We will try to obtain the information from the papers. If any domain is unclear, we will write to the authors for clarification. If clarification is not obtained after 30 days, we will assign a grading to the outcome based on the available information and the consensus between the review authors.

We will use the tool designed by the EPOC group (EPOC Risk of Bias 2009) for RCTs, which considers the following domains.

1. Was the allocation sequence adequately generated?

2. Was allocation adequately concealed?

3. Were baseline outcome measurements similar?*

4. Were baseline characteristics similar?

5. Were incomplete outcome data adequately addressed?*

6. Was knowledge of the allocated interventions adequately prevented during the study?*

7. Was the study adequately protected against contamination?

8. Was the study free of suggestion of selective outcome reporting?

9. Was the study free from other risks of bias?

      *Assessments will be made for each main outcome (or class of outcomes).

Each domain will be labelled as 'low', 'high', or 'unclear' risk of bias. The first six domains will be considered as key domains. We will summarise the overall RoB for each outcome (or class of similar outcomes) considering the assessments for these key domains in two different manners (Higgins 2011a):

• within each study across domains, each outcome (or class of outcomes) will be defined as having a ‘low risk of bias’ only if it meets all the key domains; as ‘high risk of bias’ if it demonstrates high risk of bias for one or more of them; or an ‘unclear risk of bias’ if it demonstrates unclear risk of bias for at least one key domain without any of them described as ‘high risk of bias’;

• across studies, each outcome (or class of outcomes) will be defined as having a ‘low risk of bias’ if most information is from studies at low risk of bias; as ‘high risk of bias’ if the proportion of information from studies at high risk of bias is sufficient to affect the interpretation of the results; or an ‘unclear risk of bias’ if most information is from studies at low or unclear risk of bias.

We will present our findings in a ‘Risk of bias' table and will use graphs and figures to summarise our assessments across studies.

Measures of treatment effect

Measures of effect will depend on the type of study design and type of data presented in the individual studies. For dichotomous process of care outcomes, we will report a single effect size for each type of outcome and comparison in each study. If more than one process of care outcome is reported within the same study, we will use the primary outcome as defined by the study author. If there is not a clear primary outcome available, we will calculate and use a median from all available outcomes. Further, we will perform a sensitivity analysis in which, instead of the median outcome, we will use the best outcome from each study.

We will standardise the effect size so that a positive difference between post‐intervention values is a good outcome. For each outcome and comparison in each study, we will attempt to calculate the following measures of treatment effect (separately for each study), with the 95% confidence interval (CI) and the P value*: 

• Dichotomous data (we will report outcomes in natural units): post‐intervention risks difference (RDpost).

• Continous data: post‐intervention means difference (DMpost).

*If P < 0.05 we will report the exact P value (where possible) and if P > 0.05 then we will report non‐significant (‘NS'). If there is a potential unit of analysis error, we will not quote a P value.

Unit of analysis issues

Studies with allocation to interventions at the group level

We expect that many eligible studies will be cluster designs (studies in which the unit of allocation is not a person but is instead a group of people). If these designs are included in the review, we will determine whether the data were correctly analysed: comparisons that allocate groups of participants (for example, primary care centres) should account for clustering during analysis in order to prevent 'unit of analysis' errors, resulting in artificially extreme P values and over narrow CIs (Ukoumunne 1999).

In a cluster‐RCT we will consider data to have been analysed correctly if:

1. the analysis was conducted at the same level as the allocation (i.e., at the 'cluster' level); or

2. the usual analysis was used but the sample size was reduced to its ‘effective sample size’ or the variance was inflated by the design effect; or

3. the analysis was conducted at the level of the individual but appropriate statistical correction for the clustering was performed (such as generalised estimating equations (GEE), mixed models, or multilevel models).

If we detect unit of analysis errors we will not attempt to reanalyse these data and we will report the results of the study as point estimates of the intervention effect without presentation of any statistical analysis (P values) or CIs (Higgins 2011b).

Sample size calculation

We will also assess if the unit of allocation has been taken into consideration in the sample size calculation or power calculation. In cluster designs the sample size estimate has to be inflated to take account of the cluster design (Ukoumunne 1999), so we will evaluate whether the sample size has been estimated based on the intra‐cluster correlation co‐efficient (ICC).

Dealing with missing data

We will try to carry out analyses on an 'intention‐to‐treat' principle, that is, an analysis that fulfils the next principles:

1. keeps participants in the intervention groups to which they were randomised, regardless of the intervention they actually received;

2. there is a measurement of outcome data on all participants; and

3. includes all randomised participants in the analysis (Higgins 2011b).

We will contact the primary authors to request missing data and clarification of issues. If after 30 days we cannot obtain this information, we will perform an ‘available case analysis’. An ‘available case analysis’ includes data on only those participants whose results are known, using as a denominator the total number of people who had data recorded for the particular outcome in question (the participants are analysed according to the group to which they were randomised). If possible, we will ‘re‐include’ avoidable exclusions done by the authors of the included studies (Higgins 2011b). 

We will describe missing outcomes of the included studies by reporting proportions of allocated participants for whom no outcome data were obtained (with reasons) by outcome and by study group. We will address the potential impact of the missing outcomes on the results of the included studies in the assessment of risk of bias and we will describe in the 'Discussion' section its impact on the findings of the review.

If relevant data are not directly reported in the trial reports (for example, standard deviation for continuous data), we will attempt to obtain this information by looking carefully in the report for statistics that allow its calculation (Higgins 2011c). If this is not possible, we will try to contact the primary authors. If finally we cannot obtain the information in less than 30 days, we will detail this in the data extraction template and we will try to impute this information following the suggestions provided in Higgins 2011c.

Assessment of heterogeneity

We will assess heterogeneity by:

1. preparing tables that summarise the study characteristics (types of participants, interventions, outcomes, and study design). This will allow us to examine the similarity between the studies regarding relevant factors;

2. reporting the individual effect sizes of the studies (medians and interquartile ranges (IQRs)) in tables and graphs (such as forest plots), in order to display and assess the potential disparity among the estimates and the degree of overlap among the IQRs;

3. examining the results of the Chi² test for statistical heterogeneity (a significant P value will be considered as P < 0.10);

4. examining the results of the I² statistic for the quantification of statistical heterogeneity. The I² statistic describes the percentage total variation across studies that is due to heterogeneity rather than chance. We will judge the importance of the observed value of the I² statistic depending on the magnitude and direction of effects and the strength of evidence for heterogeneity (moderate to high heterogeneity will be defined as I² statistic > 50%) (Deeks 2011); and

5. we will prepare graphs comparing effect sizes grouped according to potential effect modifiers (see Subgroup analysis and investigation of heterogeneity).

Assessment of reporting biases

We will attempt to minimize reporting bias by:

• including both published and unpublished studies;

• in the case of studies with multiple publications, by extracting data on outcomes from the publication with the most mature data; and

• not excluding studies solely on the basis of the publication language.

If sufficient studies are found, we will assess publication bias by means of a funnel plot for each outcome. We will assess funnel plot asymmetry statistically. If there is evidence of asymmetry, we will consider publication bias as only one of a number of possible explanations.

Data synthesis

We expect to find considerable heterogeneity between studies due to diversity in relevant factors, such as patient and healthcare professional populations, outcomes, strategies being evaluated, and methodological features of the study designs. Moreover, we also expect that many eligible studies will exhibit ‘unit of analysis' errors. Performing a conventional meta‐analysis using studies with unit of analysis errors will require us to make assumptions about usually unreported parameters, such as the ICCs, in order to avoid spurious precision in 95% CIs (Shojaina 2009; Higgins 2011b). For these reasons, we will not perform meta‐analysis to combine the data. 

However, we will preserve the goal of quantifying the effects associated with the identified strategies. With this aim, we will follow the approach provided in other systematic reviews of quality improvement strategies (Grimshaw 2004; Shojaina 2004a; Shojaina 2004b; Steinman 2006; Walsh 2006; Shojaina 2009). For each outcome, we will calculate the median effect size and interquartile range (IQR) across all the included studies sharing specific features of interest (Grimshaw 2004; Shojaina 2009). We will aggregate and analyse data according to the following.

• Type of strategy (EPOC Data Collection Checklist 2009) within each healthcare profession. We will also synthesise the effects of the interventions by grouping them in the following way (Grimshaw 2004):

  • single intervention compared with no intervention control (i.e., standard practice or control group that did not receive any intervention);

  • single intervention compared with another single intervention;

  • single intervention compared with a multifaceted intervention;

  • multifaceted intervention compared with no intervention control; and

  • multifaceted intervention compared with another multifaceted intervention.

• Type of outcome:

  • process of care variable, dichotomous and continuous; or

  • patient clinical outcomes, dichotomous and continuous.

The results for all comparisons will be presented using a standard method of presentation, where possible:

• • the number of comparisons with effect sizes showing a positive direction of effect;

  • the median of effect size across all comparisons;

  • the interquartile range of effect sizes across all comparisons;

  • the range of effect sizes across all comparisons;

  • the median effect size across comparisons without unit of analysis errors; and

  • the number of comparisons showing statistically significant effects.

Subgroup analysis and investigation of heterogeneity

If there are enough numbers of trials in comparisons, we will assess the impact of the following features on the effect magnitude.

• Study design: individual patient RCTs versus cluster‐RCTs.

• Sample size, we will compare the median effects across large and small studies. We will define a study as ‘large’ if its sample size is greater than or equal to the median sample size across all included studies. We will use various measures of sample size:

  • the numbers of patients (or episodes of care) without any adjustment for clustering;

  • the effective sample size after adjusting for clustering (we will look for ICCs in the published literature); and

  • the numbers of cluster units.

• Baseline adherence, we will analyse the potential impact of baseline adherence in several ways:

  • we will compare the median effect across studies with low adherence in the control group versus studies with high adherence in the control group. We will define ‘low adherence’ as lower than the median across all studies and ‘high adherence’ as greater than or equal to the median adherence across all included studies;

  • we will compare the median effects across studies with baseline adherence in the top quartile versus all others (to look for a ‘ceiling effect’); and

  • we will compare the median effects across studies with baseline adherence in the bottom quartile versus all others (to look for a ‘floor effect’).

• Risk of bias: studies with low risk of bias versus studies with high risk of bias versus studies with unclear risk of bias.

We will perform all such comparisons using a non‐parametric rank‐sum test (Mann‐Whitney). We will perform these analyses using SPSSW version 18.

Sensitivity analysis

Instead of the median outcome, we will use the best outcome from each study.

Appendices

Appendix 1. Medline search strategy

Ovid MEDLINE(R) In‐Process & Other Non‐Indexed Citations and Ovid MEDLINE(R) <1946 to Present> [May 30, 2012]

1 alcohol drinking/ (45775) 2 alcoholism/ (63390) 3 alcoholic intoxication/ (10359) 4 (alcohol$ or drinking).ti. (109691) 5 (alcoholic? or alcoholism).ab. (42165) 6 (alcohol adj3 (abus$ or addiction? or binge? or bingeing or consumption or dependen$ or excess$ or intervention? or misus$ or "overuse" or problem? or unhealthy or "use")).ab. (60038) 7 ((drinker? or drinking) adj3 (alcohol$ or "at risk" or harmful or hazard$ or "high risk" or binge$ or problem or excess$)).ti,ab. (13630) 8 ((alcohol or alcoholism or alcoholic?) adj3 (diagnosis or diagnosing or prevention or preventiv$ or preventativ$ or screening)).ab. (3420) 9 Drinking behavior/ and alcohol$.ti,hw. (538) 10 (substance related disorders/ or substance abuse detection/) and alcohol$.ab. (12883) 11 or/1‐10 [Alcoholism] (196062)

12 Primary Health Care/ (47584) 13 General Practice/ or Family practice/ or General Practitioners/ or Physicians, Family/ or Physicians, Primary Care/ or General Practice, Dental/ or Primary Care Nursing/ (77767) 14 (primary adj4 (care or healthcare or prevention)).ti,ab. (90746) 15 ((general or family) adj2 (practice? or practitioner? or physician? or doctor?)).ti,ab. (82856) 16 (routine adj2 (visit? or health care or healthcare)).ti,ab. (1499) 17 (general medical exam$ or health check? or check‐up?).ti,ab. (6674) 18 Nurse practitioners/ (14405) 19 nurse practitioner?.ti,ab. (7265) 20 Physician assistants/ (3874) 21 (physician? assistant? or doctor? assistant? or physician? extender? or feldsher?).ti,ab. (2786) 22 or/12‐19 [Primary Care/Nurse Practitioner] (225111)

23 community health nursing/ or community health services/ or community health centers/ (46884) 24 Ambulatory Care/ (33438) 25 ((ambulatory or walk‐in or neighbo?rhood or community) adj2 (clinic? or care centre or care centres or care center? or health$ centre or health$ centres or health$ center?)).ti,ab. (7715) 26 (medical setting? or healthcare setting?).ti,ab. (3513) 27 or/23‐26 [Community or other medical settings which may be frontline] (88625)

28 Practice Guidelines as Topic/ or Guidelines as Topic/ or Clinical Protocols/ or Critical Pathways/ or Guideline Adherence/ (124282) 29 (guideline? adj3 (adher$ or FOLLOW$ or implement$ or application$ or concord$ or using or introduc$ or treatment? or screening or diagnos$ or intervention? or reminder?)).ti,ab. (30148) 30 ((program? or programme or programmes) adj3 (adher$ or implement$ or application$ or concord$ or introduc$ or treatment? or screening or diagnos$)).ti,ab. (50861) 31 (protocol? adj2 (clinical or treatment or diagnos$ or screen$)).ti,ab. (18707) 32 ((clinical or treatment) adj2 algorithm?).ti,ab. (4091) 33 exp Consensus Development Conferences as Topic/ (1988) 34 ((guideline? or protocol?) adj10 (physician led or nurse‐led)).ti,ab. (65) 35 ((GUIDELINE? or PROTOCOL?) adj10 ((physician? or nurse or nurses or doctor?) adj3 (EDUCATION$ or training or teaching or learning or uptake or implement$ or adher$ or follow$))).ti,ab. (1026) 36 or/28‐35 [GL implementation, adher/Consensus Conferences etc] (213756)

37 11 and 22 [Alcoholism & PC] (4586) 38 (11 and 27) not 37 [Alcoholism & Community Care/other med settings] (1739) 39 (11 and 36) not (or/37‐38) [Alcoholism & Guideline Terms & synonyms] (3424)

40 (randomized controlled trial or controlled clinical trial).pt. or randomized.ab. or placebo.ab. or clinical trials as topic.sh. or randomly.ab. or trial.ti. (786638) 41 exp animals/ not humans.sh. (3717656) 42 40 not 41 [Cochrane RCT Filter 6.4.d Sens/Precision Maximizing] (727358)

43 controlled clinical trial/ or multicenter study/ (224810) 44 cluster.ti,ab. (92248) 45 random$.ti,ab. (597228) 46 or/43‐45 [Additional Methodological Filter Terms] (857234)

47 37 and 42 [Alcohol & PC & Cochrane RCT Filter] (571) 48 (and/37,46) not 47 [Alcohol & PC & Additional Methodological Terms] (357) 49 (38 and (or/42,46)) not (or/47‐48) [Alcoholism & Other settings & Filter] (331) 50 (and/39,42,46) not (or/47‐49) [Alcoholism & GL Terms & Filters] (276) 51 or/47‐50 (1535) [All Results]

What's new

Last assessed as up‐to‐date: 13 June 2012.

Date	Event	Description
26 June 2018	Amended	The planned review outlined in this protocol has not been successfully converted into a full Cochrane Review within established timelines and for this reason has been withdrawn by the EPOC CRG from the CDSR.

History

Protocol first published: Issue 1, 2004

Date	Event	Description
13 June 2012	Amended	New team of authors, protocol rewritten to reflect advances in Cochrane and EPOC methods
29 May 2012	New citation required and major changes	New team of authors, updated methods
3 November 2008	Amended	Converted to new review format.

Contributions of authors

Teresa Sanz Cuesta (TSC): guarantor. Co‐ordinated the protocol development. Identified references for the protocol background. Designed, drafted and wrote the protocol. Organised retrieval of papers. Made an intellectual contribution to the protocol and provided a clinical perspective to the manuscript. Approved final version of the protocol prior to submission.

Jesús López Alcalde (JLA): co‐ordinated the protocol development. Designed, drafted and wrote the methodological sections of the protocol. Supported designing, drafting and writing of the remaining sections of the protocol. Approved final version of the protocol prior to submission.

Isabel del Cura González (ICG), Jesús Martín Fernández (JMF), Elisa Ceresuela Wiesman (ECW) y Tomás Gómez Gascón (TGC): edited the protocol. Identified references for the protocol background. Made an intellectual contribution to the protocol and provided a clinical perspective to the manuscript. Approved final version of the protocol prior to submission.

Iván Solà Arnau (ISA): designed, drafted and wrote the search methods of the protocol. Supported designing, drafting and writing of the remaining sections of the protocol.

Rest of authors: made an intellectual contribution to the protocol and provided a clinical perspective to the manuscript. Approved final version of the protocol prior to submission.

Sources of support

Internal sources

• Unidad de Apoyo a la Investigación. Gerencia Adjunta de Planificación y Calidad. Gerencia de Atención Primaria de Madrid, Spain.

• UETS, Health Technology Assessment Unit. Agency Laín Entralgo (Cochrane Collaborating Centre), Madrid, Spain.

External sources

• This systematic review forms part of a wider project about the effectiveness of strategies to implement clinical practice guidelines in Primary Care (‘Efectividad de las estrategias de implementación de guías de práctica clínica en atención primaria’, Grant PI08/90777 "Acción Estratégica de Salud" (2008), Plan Nacional de Investigación Científica, Desarrollo e Innovación Tecnológica 2008‐2011(ISCIII)), Spain.

Declarations of interest

None known

Notes

The planned review outlined in this protocol has not been successfully converted into a full Cochrane Review within established timelines and for this reason has been withdrawn by the EPOC CRG from the CDSR.

Withdrawn from publication for reasons stated in the review