Interventions Encouraging the Use of Systematic Reviews in Clinical Decision-Making: A Systematic Review

Abstract

Background

Systematic reviews have the potential to inform clinical decisions, yet little is known about the impact of interventions on increasing the use of systematic reviews in clinical decision-making.

Purpose

To systematically review the evidence on the impact of interventions for seeking, appraising, and applying evidence from systematic reviews in decision-making by clinicians.

Data Sources

Medline, EMBASE, CINAHL, Cochrane Central Register of Controlled Trials, and LISA were searched from the earliest date available until July 2009.

Study Selection and Data Extraction

Two independent reviewers selected studies for inclusion if the intervention intended to increase seeking, appraising, or applying evidence from systematic reviews by a clinician. Information about the study population, features of each intervention, methods used to measure the use of systematic reviews and those used to measure professional performance or health care outcomes, existence and use of statistical tests, study outcomes, and comparative data were extracted.

Data Synthesis

A total of 8,104 titles and abstracts were reviewed, leading to retrieval of 189 full-text articles for assessment; five of these studies met all inclusion criteria. All five studies reported on professional performance behavior; none reported on patient health outcomes. One study reported positive outcomes in improving preventive care. Three studies focused on obstetrical care, with two reporting no impact on professional practice change, and one study reporting increases in the use of prophylactic oxytocin and episiotomy. One study found no improvement in the sealant rate of newly erupted molars among dentists in Scotland.

Limitations

The small number of studies available for examination indicates the difficulty in summarizing and identifying key aspects in successful strategies that encourage clinicians to use systematic reviews in decision-making. Other concerns lay in selective reporting and lack of blinding during data collection.

Conclusions

The limited empirical data render the strength of evidence weak for the effectiveness and types of interventions that encourage clinicians to use systematic reviews in clinical decision making.

INTRODUCTION

Ideally, clinical decisions would be made using the best available evidence. However, simple dissemination of the results of research does not guarantee implementation.¹ It remains a universal challenge to implement research findings that show benefit or reduce harm.^2-5

Despite the stated purpose of facilitating decision-making and the concerted effort to enhance and clarify the methods of systematic reviews,⁶ studies suggest their use in clinical settings is not widespread.^7,8 In a systematic review of information seeking practices among clinicians, Dawes and Sampson⁹ found that textbooks were the most frequent source of information, followed by advice from colleagues. Surveys reported in 2001 and 2002 that physicians had limited use or awareness of the Cochrane Library and systematic reviews.^10-12 In the United Kingdom, it was found that only 10% regularly referred to the Cochrane Library, and 52% had never heard of it.^10,12 Similarly, in New Zealand only 15% had ever used it.¹¹ In examining nurses’ use of knowledge sources in clinical practice, it was found they preferred to use knowledge gained through personal experience and interactions with co-workers rather than journal articles of any type.¹³ Concurrently, examples of misuse of therapies indicate systematic reviews are not regularly used to inform decision-making in clinical settings.³ To our knowledge, no systematic reviews of studies exist that examine interventions that help health practitioners use systematic reviews in making decisions in clinical settings. The review was carried out in two stages: (1) a formal scoping review to understand the extent to which evidence from systematic reviews is sought, appraised, understood, and used to inform decision-making in four key areas: clinical practice, health systems management, public health, and policy making; (2) a systematic review to determine the impact on professional performance and health care outcomes of interventions for seeking, appraising, and applying evidence from systematic reviews (as a source document) in decision-making by clinicians, which is reported in this manuscript.

METHODS

Data Sources and Searches

The databases of Medline (1950 to July 2009), EMBASE (1980 to July 2009), CINAHL (1982 to July 2009), Cochrane Central Register of Controlled Trials (CENTRAL) (to July 2009), and LISA (Library and Information Science Abstracts) (1969 to July 2009) were searched using the terms systematic review, meta-analysis, evidence synthesis, methodologic review, and quantitative review combined with implement, use, utilize, seek, retrieve, appraise, and apply. No language restrictions were placed on the search strategy. The grey literature was searched after identifying key websites and search engines. Reference lists of all papers and relevant reviews were identified, and authors of relevant papers were contacted regarding any further published or unpublished work.

Study Selection

We included all study designs except qualitative studies. All health-care providers, including physicians, nurses, and allied health professionals (e.g., physiotherapists, speech pathologists, social workers, pharmacists) involved in providing direct patient care were included. Studies that included participants in training programs who are responsible for patient care (residents, fellows, and other pre-licensure health-care professionals) were eligible for inclusion. Allied health professionals not involved with direct patient care (e.g., medical transcriptionists), students, and persons in undergraduate training programs were not considered for this review. Any study that examined interventions intended to increase seeking, appraising, or applying evidence from systematic reviews by a clinician was included. The use of products or tools derived from systematic reviews (e.g., guidelines, clinical pathways) was not considered as the focus was the use of systematic reviews. Interventions designed to enhance the uptake of evidence from systematic reviews were included. Primary outcomes of interest were any objective measure of professional performance (e.g., prescribing patterns, use of diagnostic tests) or health outcomes for patients (e.g., adverse events, return visits, length of stay, decrease in admissions). Measures of health-care providers’ satisfaction, knowledge, or attitudes were systematically collected as a secondary outcome.

Data Extraction and Quality Assessment

Standardized data abstraction forms were developed and pilot tested by the review team using the protocol to guide primary and secondary outcomes. The following information was extracted from each article: setting, location of care, country, unit of allocation, clinical area addressed, type of health-care professional (e.g., physician, nurse, physiotherapist, student nurse, resident), frequency and timing of the intervention, duration of intervention, format of the intervention (e.g., web-based, person-to-person contact), known effectiveness of the intervention for changing of health-care professional behaviors (e.g., evidence-based intervention), nature of the intervention (e.g., training, mode of payment, team approach), number of components included in the intervention, source and authors of the intervention (e.g., professional organization, governmental agency, health professionals training schools), mode of delivery (e.g., individuals or groups), and reliability and validity testing of outcome measurement tools and adherence (e.g., withdrawals, drop-outs). Two reviewers independently assessed each study and undertook data abstraction directly from primary studies. Disagreements were discussed until consensus was achieved. A third reviewer was available if consensus could not be reached. Authors were contacted for missing data or when clarification was required (e.g., intervention not described in sufficient detail).

Two independent reviewers assessed the methodological quality of all studies that were included for data abstraction. Any discrepancies in ratings were resolved by discussion. Reviewers were not blinded to study author, institution, or journal, as evidence indicates that little benefit is achieved through blinding.^14,15 The criteria described in section 6.4 of the Data Collection Checklist from the Cochrane Effective Practice and Organisation of Care Group (available at: http://www.epoc.cochrane.org) were used. The criteria used to assess randomized trials were concealment of allocation, follow-up of professionals, follow-up of patients or episodes of care, blinded assessment of primary outcome(s), baseline measurement, reliable primary outcome measure(s), and protection against contamination.

RESULTS

Initial searches of electronic databases identified 12,737 records for the scoping review. Out of 8,104 de-duplicated titles and abstracts, nine studies were deemed relevant in our review.^16-24 Review of full text articles led to exclusion of three studies because the intervention was not relevant^22-24 and one study because it was a feasibility study.²¹ Five relevant randomized controlled trials met the full inclusion criteria (Fig. 1).^16-20

Figure 1.

Search and selection of included studies.

Five cluster randomized controlled trials (RCT) examining educational interventions for seeking, appraising, and applying evidence from systematic reviews in decision-making by clinicians were identified (Tables 1).^16-20 A meta-analysis of study outcomes was not possible due to the heterogeneity in the format of the interventions, the settings, and clinical areas being addressed.

Table 1.

Description of Studies Included in the Systematic Review

Source	Participants and setting	Content area	Time	Intervention	Measures	Study outcomes	Quality assessment
Lemelin et al., 2001 (16)	Physicians	Preventive care	Clinician time commitment:	Multi-faceted intervention delivered by nurse facilitators trained to improve prevention in primary care	1. Folic acid supplementation	Three outcome measures: (1) overall index of preventive performance, (2) up-to-datedness index, (3) inappropriateness index was calculated based on mean percentage of eligible patients receiving 8 recommended preventive maneuvers and 5 inappropriate preventive maneuvers	Sequence generation and allocation concealment not adequately described
	Health service organization: primary care		Average = 33 meetings approximately 1.75 h in length over 18 months		2. Smoking cessation counseling	All 3 measures reported as showing absolute improvement in intervention group	Possible reporting bias and selection bias
	n = 100 Health service organizations in Canada		Follow-up period: 9, 15, 18 months		3. Hypertension treatment	Intervention described as effective in modifying physician practice patterns and significantly improving preventive care performance
					4. Mammography in women aged 50–69 years
					5. STD screening
					6. Papanicolaou smear
					7. Influenza vaccination
					8. Blood pressure measurement
					9. Proteinuria screening
					10. Blood glucose screening
					11. Prostate-specific antigen testing
					12. Chest radiography
					13. Mammography in women aged 40–49 years
Gülmezoglu et al., 2007 (17)	Physicians, midwives, interns, students	Obstetrics	Clinician time commitment: maximum = 3 workshops; length of workshops not reported	Multi-faceted:	1. Social support during labor	No increase shown in 3 of 10 measures (iron/folate supplementation, uterotonic use after birth, and breastfeeding on demand)	Mexico: Contamination due to some physicians working in more than one hospital
	Hospitals: maternity wards in Mexico and Thailand		Follow-up period: 10–12 months	▪ Organizational buy-in	2. MgSO4 for eclampsia	External cephalic version could not be measured	Thailand: Incomplete data from control group related to process outcomes
	n = 40 Hospitals			▪ Provision of materials	3. Corticosteroids to women with preterm birth	No significant differences in the remaining 6 practices (use of labor companionship, magnesium sulfate use for eclampsia, corticosteroids for women delivering before 34 weeks, vacuum extraction, selected episiotomy, use of antibiotics at caesarean section)	Both countries: selective reporting as investigators unable to measure one out of ten measures
				▪ Use of facilitators	4. Selective episiotomy	Reproductive Health Library: awareness and use increased substantially after the intervention in both countries
				▪ Print materials	5. Uterotonic use after birth*
				▪ Three interactive workshops on using WHO Reproductive Health Library	6. Breastfeeding on demand*
					7. External cephalic version†
					8. Iron/folate supplementation*
					9. Antibiotic use at caesarean section
					10. Vacuum extraction for assisted birth
Wyatt et al., 1998 (18)	Physicians, midwives	Obstetrics	Clinician time commitment:	Single informal educational visit by an obstetrician including:	1. Perineal suturing with polyglycolic acid	Rates of ventouse delivery reported as increasing significantly in intervention units but not in control units; however may have been regression towards the mean	Risk of contamination in control group
	Hospitals: obstetric units		1 Meeting lasting 1.5–3 h in length	▪ Discussion of evidence-based obstetrics	2. Ventouse delivery [vacuum extraction]	There was no difference between the intervention and control units in uptake of other practices
	n = 25 Obstetric units in the UK		Follow-up period: 9 months	▪ Guidance on implementation	3. Prophylactic antibiotics in caesarean section
				▪ Donation of Cochrane database and other materials	4. Steroids in preterm delivery
Althabe et al., 2008(19)	Physicians, residents, midwives	Obstetrics	Clinician time commitment: minimum = 6 days to complete workshops alone	Multi-faceted:	1. Rates of prophylactic use of oxytocin	Rate of use of prophylactic oxytocin increased from 2.1% to 83.6% in intervention group, 2.6% to 12.3% in control group (p = 0.01 for the difference in changes)	Missing data acknowledged but no explanations given or assessment of potential impact
	Hospitals: obstetric units		Follow-up period: 12 months after the end of intervention	▪ Workshops	2. Rates of episiotomy in singleton vaginal deliveries	Rate of use of episiotomy decreased from 41.1% to 29.9% in intervention hospitals, remained stable at control hospitals at 43.5% and 44.5% (p < 0.001 for the difference in changes)	Data collectors not blinded to whether hospital was in intervention or control group
	n = 19 Hospitals in Argentina and Uruguay			▪ Educational visits
				▪ Opinion leaders
				▪ Audit and feedback
				▪ Reminders
				▪ Computers installed with resources, e.g., WHO Reproductive Health Library
Clarkson et al., 2008 (20)	Dentists	Dentistry	Clinician time commitment: maximum = 1-day workshop for dentists in ‘education’ arms of study	1-day workshop given to dentists randomized to ‘education’ arm and ‘education + fee’ arm of study	Fissure-sealing of newly erupted molars on 12-14 year olds	Cluster level analysis shows a significant increase in sealant treatment in ‘fee’ arm compared to other arms (adjusted risk difference, 9.8%; CI 1.8%–17.8%)	Outcome measure depended on accuracy of dentist's reports
	n = 133 Dentists in Scotland		Follow-up period: 12 months				Dentists not attending 1-day workshop retained in ‘education’ arm. Number of non-attendees not reported
							Data collected from 133 practitioners fell below 150 reported as necessary in estimation of main effects

*Ceiling effect (0.70%). ^†Unable to measure

Quality Assessment Results

Quality assessments of the studies indicate that selective reporting and other problems that could put them at a risk of bias were identified as sufficient to affect interpretation of results. All studies reported follow-up of professionals sufficiently.^16-20 Allocation concealment was not done for two studies^19,20 and was unclear for one study.¹⁶ For selective reporting, Gülmezoglu et al.¹⁷ were not able to report on one practice at follow-up due to the inability to measure. Survey data for controls at follow-up were deemed to be contaminated and thus not reported. Some doctors were working in more than one of the participating hospitals in Mexico, and this may have also contributed to contamination.¹⁷ Freedom from bias was unclear for Lemelin et al.¹⁶ due to inadequate information about the practices that gave informed consent to participate in the study, but later declined (19 out of 100 sites). Contact with the authors confirmed that the telephone survey was restricted by language, and the survey was terminated if the patient did not speak English, thus introducing another source of bias.

Two studies were identified following assessment for methodological quality as being of low quality.^19,20 Concerns for both studies lay in the lack of rigor related to data collection.^19,20 Data collectors were not blinded to whether a hospital was part of the intervention or control group in the study performed by Althabe and colleagues.¹⁹ Clinicians were aware of the treatment of interest and reported their own outcomes. which were not corroborated or assessed for accuracy in the trial conducted by Clarkson et al.²⁰

Participants and Settings

Three cluster RCTs were based in a hospital obstetric care setting. Wyatt et al.¹⁸ recruited doctors and midwives from 25 obstetric units in two regions in the UK. Gülmezoglu and colleagues,¹⁷ building on the earlier work of Wyatt, recruited doctors and midwives from 22 hospitals in Mexico and 18 hospitals in Thailand. Althabe et al.¹⁹ conducted a trial in 17 public maternity hospitals in Argentina and 2 in Uruguay with physicians, residents, and midwives. One hundred thirty-three dentists were examined by Clarkeson et al.²⁰ in Scotland, who were working in areas of a post-code-based system defining deprivation looking at their rates of fissure sealing of newly erupted molars. Lemelin et. al.¹⁶ examined primary care physicians located in 46 health service organizations in Canada, concentrating on the topic of preventive care.

Interventions

Wyatt and colleagues¹⁸ used a single informal educational visit to obstetric units in the intervention group lasting between 1½–3 h in length. During these visits an obstetrician discussed with the senior obstetrician and midwife the principles of evidence-based obstetrics and how to find and select Cochrane reviews in order to inform clinical practice.¹⁸ Afterwards, a copy of the Cochrane database was donated to the unit for their use.¹⁸ Gülmezoglu et al.¹⁷ ran three interactive workshops encouraging participants to use the Cochrane Reproductive Health Library, along with other strategies such as the provision of computer equipment in order to access the systematic reviews and promotional brochures publicizing the Library. Similarly, Althabe and colleagues¹⁹ implemented a multi-intervention strategy including interactive workshops that ran over a 5-day period, along with computer equipment and a copy of the Cochrane Reproductive Health Library. The approach evaluated by Lemelin et al.¹⁶ involved trained facilitators who organized meetings to introduce primary care staff to audit and feedback, academic detailing, and reminder systems. An average of 33 meetings took place over the 18-month study period with each meeting lasting about 1.75 h. Contact with the authors confirmed that one component focused on academic detailing and education materials that encouraged clinicians to seek and use systematic reviews to improve preventive care; however, it did not specifically provide access to systematic reviews. Clarkson and colleagues²⁰ studied dentists, and those randomized into the ‘education arm’ of their 2×2 factorial design were invited to a 1-day workshop with the goal to provide skills to implement an evidence-based approach to clinical practice and raise awareness of research methods in primary care. The other implementation strategies of this study were remuneration, and a combination of education and remuneration. All five trials included a control group that received no intervention.

All studies used multiple strategies in their intervention, but only Gülmezoglu et al.,¹⁷ Lemelin et al.,¹⁶ and Althabe et al.¹⁹ explicitly describe their interventions as multi-faceted.^16,17 Gülmezoglu et al.¹⁷ and Wyatt et al.¹⁸ focused on ‘knowledge access’ with both interventions promoting the singular goal of helping clinicians use systematic reviews with supportive measures such as the provision of tools (e.g., computers, printers, copy of the Cochrane Reproductive Health Library). Clarkson et al.²⁰ offered an interactive workshop that encouraged the use of systematic reviews as one component of the educational session, but did not provide supportive materials. In contrast, the goal for the intervention evaluated by Lemelin et al.¹⁶ was to improve prevention in primary care, with one strategy out of the seven helping clinicians use systematic reviews to make decisions, and Althabe et al.¹⁹ focused on facilitating the adoption of evidence-based practices with one strategy out of eight offering a component that encouraged clinicians to seek out systematic reviews for decision-making.

Effect of the Intervention

Wyatt and colleagues¹⁸ measured four clinical markers [rates of perineal suturing with polyglycolic acid, ventouse delivery (vacuum extraction), prophylactic antibiotics in caesarean section, and steroids in preterm delivery] using data collected through chart audits before and 9 months after an educational visit. Results indicate no difference between intervention and control units in uptake of these practices.¹⁸ Rates for ventouse delivery were reported as significant in intervention units between baseline and follow-up (risk ratio for failure to use ventouse delivery: 0.68; 95% CI 0.59–0.78), but not in control units (risk ratio for failure to use ventouse delivery: 0.96; 95% CI 0.82–1.12).

In a similar approach, Gülmezoglu et al.¹⁷ selected ten clinical practices recommended in the Cochrane Reproductive Health Library (such as antibiotic use for caesarean section and social support during labor). Data were collected at baseline, then again after 10–12 months.¹⁷ In both study sites (Mexico and Thailand), there were no statistically significant differences between the intervention and control groups for the outcomes studied.¹⁷ The largest improvement reported was a higher reduction in the use of episiotomy in the intervention hospitals in Thailand, reported as approaching statistical significance (difference in adjusted mean rate =-5.3%; 95% CI: -0.1% to 10.7%).¹⁶ Gülmezoglu et al.¹⁷ also conducted a survey to gather information about awareness and use of the Cochrane Reproductive Health Library reporting that awareness (baseline to follow-up: 24.8%–65.5% in Mexico and 33.9%–83.3% in Thailand) and use (baseline to follow-up: 4.8%–34.9% in Mexico and 15.5%–76.4% in Thailand) increased substantially.

In the third cluster randomized trial that focused on obstetrics, Althabe and colleagues¹⁹ collected data at baseline, at the end of the 18-month intervention, and at follow-up 12 months after the end of the intervention on two primary outcomes (rates of episiotomy, rates of prophylactic use of oxytocin). Post-intervention, the rate of use of prophylactic oxytocin increased (baseline to follow-up: 2.1%–83.6% at intervention hospitals, 2.6% to 12.3% at control hospitals; p = 0.01 for the difference in changes). The rate of use of episiotomy decreased at intervention hospitals, but remained stable at control hospitals (baseline to follow-up: 41.1%–29.9% at intervention hospitals, baseline and follow-up: 43.5% and 44.5% at control hospitals; p < 0.001 for the difference in changes). Although the absolute difference in rate changes remained increased in the intervention hospitals as compared to the control hospitals at follow-up 12 months after the intervention was completed, it did not reach statistical significance. All results were reported for both countries, Argentina and Uruguay, combined.

Lemelin and colleagues¹⁶ identified 13 preventive maneuvers and scored practices using an index of preventive performance. Baseline data were collected and follow-up data at 9, 15, and 18 months, using chart audits. Preventive performance was identified as similar at baseline between intervention and control groups, with an absolute improvement in the intervention group of 11.5% (p < 0.001) at 18-month follow-up (risk ratio: 1.35). Improvement in the preventive performance index is reported at all data collection points (9 months, 15 months, and 18 months) over time for the intervention; however, this is not tracked to specific preventive measures.

Clarkson et al.²⁰ investigated the effects of financial incentive and of educational sessions on sealant treatment on newly erupted molars of 12–14 year olds. Data were collected 12 months post-intervention. Cluster-level analysis showed a significant increase in sealant treatment in the fee arms (fee alone, or fee + education session) compared with the other arms (adjusted risk difference, 9.8%; CI 1.8%–17.8%).

DISCUSSION

To our knowledge, this is the first systematic review of the evidence on the impact of interventions for seeking, appraising, and applying evidence from systematic reviews in decision-making by clinicians. The review of five studies revealed an absence of research on interventions that encourage clinicians to use systematic reviews in decision-making. Interventions focusing on enhancing the use of systematic reviews ranged from one meeting lasting as little as 1.5 h in one study,¹⁸ to being delivered over an average of 33 meetings lasting approximately 1.75 h each over an 18-month time span.¹⁶ Wyatt and colleagues report increased rates of use of ventouse delivery in intervention units between baseline and follow-up; however, since both intervention and control groups showed similar performance at follow-up, the authors suggest that this may be due to regression to the mean.²² Both Lemelin et al.¹⁶ and Althabe et al.¹⁹ indicated positive outcomes in changing practice; however, neither examined clinician use of systematic reviews in decision-making as its sole focus, thus creating difficulty in determining the extent to which this component can account for changes in professional behavior, if at all.^16,19 No studies were identified that examined health outcomes for patients.

It is not possible to know if associations in one clinical area can provide understanding for other clinical areas. Learning from Wyatt and colleagues¹⁸ and their use of a single educational visit along with support materials, Gülmezoglu and colleagues¹⁷ similarly focused on pregnancy and childbirth, but used an alternate strategy of interactive workshops with support materials. One significantly different feature between the two studies is that Gülmezoglu and colleagues completed their work in developing countries.¹⁷ Although both authors confirm a lack of success in the use of either educational strategy, these studies build upon our knowledge within the area of obstetrics by providing the opportunity to examine outcomes from different interventions. Althabe and colleagues primarily were encouraging clinicians to develop and implement guidelines; however, a component of their workshops specifically encouraged practitioners to use systematic reviews in decision-making as well as providing the tools to support this (e.g., Cochrane Reproductive Health Library).¹⁹ Similar to Lemelin et al.¹⁶, it is not possible to assess the impact of this one component on its own.

All five studies highlight several considerations for the evaluation of systematic reviews and their importance in evidence-based health care. Two studies focused on the goal of ‘knowledge access’ in obstetrical care.^17,18 Both sets of researchers did not inform staff of the practices selected as outcomes, but rather focused on giving them the knowledge and skills to use available tools (such as the Cochrane Library or the Reproductive Health Library), and extract the information that they would identify as important, and implement changes. Gülmezoglu et al.¹⁷ note this may present challenges as there is no prescriptive guidance for translating messages from systematic reviews into concrete practice changes that might be recognized in other tools, such as algorithms or clinical practice guidelines. Although Lemelin et al.¹⁶ revealed their interest in changing preventive practices to the intervention group and were able to show positive outcomes, the limitation of having this one strategy embedded within a multi-component intervention does not allow us to determine the effectiveness of this approach on its own.

Several limitations in this review should be considered. First, the literature in this area is poorly indexed. This challenge was acknowledged in the choice to conduct a scoping review as a strategy to understand the overall state of research activity in the area of the use of systematic reviews in health-care decision-making. Scoping reviews are often undertaken when an area has little published research available or the area is poorly understood.²⁵ The search strategy for the scoping review allowed for a very broad search and examination of over 8,000 articles. The small number of studies available for final assessment indicates the difficulty in summarizing and identifying key aspects in successful strategies that encourage clinicians to use systematic reviews in decision-making. The limited empirical data render the strength of evidence weak in relation to the effectiveness and the types of interventions that encourage clinicians to use systematic reviews. Second, this review is limited by the reports of methods from the included studies. When additional details were needed for data abstraction, authors of primary studies were contacted.

This review found five relevant studies that provide limited evidence that the interventions outlined changed professional behavior. Overall, there is insufficient evidence to support or refute interventions for seeking, appraising, and applying evidence from systematic reviews in decision-making by clinicians. Considerations for future research include examining the circumstances and contexts under which systematic reviews are most effective. This includes how systematic reviews are accessed under normal settings, where they are used (e.g., beside, office), when they are used (e.g., before, during, or after a consultation), and the specific characteristics that make systematic reviews easy to use in terms of presentation and format of information.

Contributors: SES created the study concept and design. LP, AM, and SES constructed and refined the search strategy. SES and LP acquired the data. Analysis and interpretation of the data were completed by LP, KM, and SES. Drafting of the manuscript and critical revision for important intellectual content was done by LP, KM, SS, MD, AM, and SES. LP wrote the final report and is the guarantor for the paper. All authors contributed to the final paper.