Strategies for Reducing Regional Variation in the Use of Surgery A Systematic Review

Abstract

Objective

To review the literature evaluating the effect of practice guidelines and decision aids on use of surgery and regional variation.

Background

The use of surgical procedures varies widely across geographic regions. Although practice guidelines and decision aids have been promoted for reducing variation, their true effectiveness is uncertain.

Methods

Studies evaluating the influence of clinical practice guidelines or consensus statements, shared decision making and decision aids, or provider feedback of comparative utilization, on rates of surgical procedures were identified through literature searches of Ovid MEDLINE, EMBASE, and Web of Science.

Results

A total of 1946 studies were identified and 27 were included in the final review. Of the 12 studies evaluating implementation of guidelines, 6 reported a significant effect. Those examining overall population-based rates had mixed effects, but all studies evaluating procedure choice described at least a small increase in use of recommended therapy. Three of 5 studies examining the effect of guidelines on regional variation reported a significant reduction after dissemination. Of the 15 studies examining decision aids, 5 revealed significant effects. Many studies of decision aids reported decreases in population-based procedure rates. Nearly all studies evaluating the impact of decision aids on procedure choice reported increases in rates of less invasive procedures. Only one study of decision aids assessed changes in regional variation and found mixed results.

Conclusions

Both practice guidelines and decision aids have been proven effective in many clinical contexts. Expanding the clinical scope of these tools and eliminating barriers to implementation will be essential to further efforts directed toward reducing regional variation in the use of surgery.

Studies of surgical variation have shown that a patient’s likelihood of operation often depends as much on where one lives as his or her clinical condition.^1–4 According to data from the Dartmouth Atlas of Healthcare, a patient’s chance of undergoing cardiovascular, oncologic, orthopedic, and other procedures can vary 3- to 10-fold across geographic areas.⁵ Although determinants of regional variation in the use of surgery are debated and likely multifactorial, all agree that this issue is particularly relevant given the current national dialogue regarding health care reform and constraining health care costs.⁶ More importantly, the presence of wide regional variation in the use of surgery implies that many patients are being undertreated or overtreated.

The optimal strategies for reducing regional variation in the use of surgery remain unclear, however. Previous research suggests that the use of surgery varies in large part as a result of clinical uncertainty, which may in turn reflect gaps in current scientific knowledge or differences in how surgeons apply evidence.^7–9 In this context, policymakers, professional organizations, and other stakeholders have pushed to disseminate more evidence-based practice guidelines and consensus statements to facilitate evidence-based clinical decision making.^9–12 Even in the presence of sound clinical evidence, regional variation can occur as a result of inconsistent incorporation of individual patient preferences in surgical decisions.^13–16 As a result, the use of decision aids and similar tools has been promoted to help patients make more informed decisions, and possibly reduce regional variation.^17–20

Despite the conceptual appeal practice guidelines and decision aids may have for reducing surgical variation, their true effectiveness remains unclear. Studies evaluating these tools have been based on diverse patient populations, heterogeneous methods, and disparate measures of effectiveness. To better synthesize the literature in this area, we performed a systematic review of the effect of practice guidelines and decision aids on the use of surgery.

METHODS

Search Strategy

We performed a thorough and structured literature review of published and unpublished articles using the electronic databases MEDLINE (Ovid) (1946, November 2012), EMBASE (1946, November 2012), and Web of Science: Conference Proceedings Citation Index—Science (1990, November 2012). The search strategy was devised with the assistance of a research librarian specialized in the surgical literature. A broad literature search with explosion was conducted on the aforementioned databases using keywords and MeSH (Medical Subject Headings) terms from seminal articles. Results from this preliminary search were reviewed, and an iterative process was used to refine the search strategy over multiple subsequent searches.

The final search included 3 domains of MeSH terms and key words combined using “AND,” whereas each domain was created using “OR.” The first domain included terms to capture articles discussing surgery and surgical procedures, the second included terms to identify the specified strategies for intervention, and the third captured articles reporting surgery rates within a population or study cohort. Both experimental and observational studies were included, and the search was limited to English language and studies of humans. Letters and editorials were excluded (see Supplemental Digital Content Appendix 1, available at http://links.lww.com/SLA/A459, for the full search). Additional searches using Google Scholar and the Cochrane Database of Systematic Reviews were performed to enhance the results by including additional gray literature. Finally, comprehensive forward and backward bibliography searches were completed on all articles from the database search that were included in this review. The flow diagram of the search and systematic review protocol is shown in Figure 1.

FIGURE 1.

Flow diagram of selection process and systematic review protocol.

Initial Review and Study Inclusion

Two investigators reviewed the titles and abstracts of all studies resulting from the initial database search. Articles were included in this review if they evaluated the rate of a surgical procedure before and after implementation or dissemination of clinical practice guidelines or consensus statements, shared decision-making tools and decision aids, or provider feedback detailing comparative rates of utilization. These were selected a priori, as they represent strategies directly related to clinical decision making and practice. Although other strategies of influence exist (eg, financial incentives, insurance models, and policy), adding these was felt to be beyond the scope of a single systematic review. Studies evaluating nonsurgical procedures such as diagnostic imaging or tests, medical (eg, endoscopy or cardiac catheterization), or dental (eg, wisdom tooth extraction) procedures were excluded. Studies focused on labor or cesarean section rates were also excluded, as this topic has been extensively studied and reviewed.^21,22 Single-institution experimental trials were considered for inclusion, but single-institution observational studies were excluded.

To ensure all possibly relevant studies were considered for inclusion, every study with uncertain eligibility at title and abstract review was retained for further examination. Full-text documents were obtained for selected articles, and then 2 investigators independently reviewed the full text of each article to determine final inclusion in the systematic review. A simple kappa statistic (κ) was calculated to determine the level of agreement between investigators regarding final inclusion during the full-text review.^23,24 Disagreements were resolved by the input of a third investigator.

Data Extraction, Summary Measures, and Analysis

Two investigators independently recorded data from included studies on a structured data extraction form. Extracted data included study year, location, design, size, type of procedure, intervention used, setting, results, and study conclusions. Study authors were contacted to obtain pertinent data if it was not reported in the full-text article. For observational studies, the reported rate of surgery before and after implementation of the intervention (and associated statistical significance) was used as the primary summary measure of this review. For randomized-controlled trials, the reported rate of surgery in experimental and control groups after the intervention was used as the primary summary measure. Measures of regional variation were recorded as secondary summary measures of this review. Given the extensive heterogeneity present between study designs (experimental and observational) and measures of effect (population-based rates, proportion of cohort, odds ratios, mean annual changes in rates), meta-analyses were not performed,^25,26 and statistical evaluation of publication bias was limited to the subset of included randomized controlled trials.²⁷ The Harbord test was applied to this subset to assess for small-study effects, and a funnel plot was visually inspected for asymmetry.²⁸ To minimize risk of publication bias in the cohort of observational studies, a thorough search of the gray literature was conducted, and throughout the review protocol careful consideration for inclusion was given to both positive and negative observational studies.

Quality Assessment

Widely established quality assessment scales were used to assess for potential bias. Two investigators independently evaluated the quality of each included study using a standardized form, and any discrepancies were resolved by a third investigator. Randomized-controlled trials were assessed using the Cochrane Collaboration’s tool for assessing risk of bias to evaluate the quality of studies in the categories of random sequence generation, allocation concealment, blinding of participants, personnel, and outcomes, outcome reporting, and other risks of bias.²⁹ Studies were considered low quality if they exhibited a high or unclear risk of bias in more than 3 major categories. Observational studies were assessed using the Newcastle-Ottawa Scale to evaluate methodological quality over the domains of selection, comparability, and outcome assessment.³⁰ Studies were considered low quality if they did not meet criteria in 3 or more major categories.

Throughout the systematic review protocol, we carefully adhered to the Cochrane Handbook for Systematic Reviews.³¹ Both the PRISMA statement³² for systematic reviews and meta-analyses of randomized-controlled trials and the MOOSE guidelines³³ for meta-analyses of observational studies were strictly followed.

RESULTS

Characteristics of Included Studies

The combined database and literature search identified 1946 studies, of which 76 qualified for full-text evaluation and 33 met inclusion criteria (Fig. 1). The k statistic revealed nearly excellent agreement between investigators after full-text review (87.8% agreement, κ = 0.733, P < 0.001). Seven of the included articles did not report specific rates of the procedure studied, and each corresponding author was contacted for additional information. Three articles were excluded because the authors confirmed the specific data of interest was not available,^34–36 and 1 study was excluded after multiple failed attempts to contact the author.³⁷

The majority of selected studies focused on clinical guidelines or decision aids. Only 2 of the selected studies evaluated the use of provider feedback detailing comparative utilization,^38,39 and thus were excluded from formal systematic review. Characteristics of the included studies are detailed in Tables 1 and 2. Twelve studies evaluated the dissemination of practice guidelines or consensus statements, and 15 examined the use of decision aids and shared decision-making tools on rates of surgical procedures.

TABLE 1.

Summary of Studies Evaluating Dissemination of Practice Guidelines or Consensus Statements

Author/s, Year	Study Design	Years Studied	Study Population	Location	Procedure	Quality	Practice Guideline or Consensus Statement	Summary of Findings	Statistically Signific ant Effect
Studies evaluating effects on population-based rates of surgery
Black and Hutchings40 2002	E	1975–1998	Children aged 10 or younger in 13 English health districts	United Kingdom	Myringotomy or tympanostomy tube insertion	Low	1992 NHS Effective Health Care bulletin	The yearly decline in mean annual procedure rate increased from 1.6% to 10.1% in NHS hospitals during the study period.	Yes
Brownell42 2002	E	1994–1999	Children aged 19 or younger in Manitoba	Canada	Tonsillectomy, adenoidectomy	High	1995 Manitoba Clinical Guidelines and Analysis Program: Tonsillectomy Review Panel	Overall procedure rates decreased significantly after guideline release in all regions studied, but rates in some regions subsequently increased.	Mixed
Mason et al41 2001	E	1989–1996	Children aged 15 or younger in England	United Kingdom	Tympanostomy tube insertion	Low	1992 NHS Effective Health Care bulletin	In the 3 years before guideline distribution there was a small quarterly procedure rate increase. This increase subsequently decreased in the 4 years after implementation. Overall procedure rates decreased from 2.1 to 1.4 per 1000.	Yes
Rob et al43 2004	RC	1981, 1983, 1986, 1988–1999	Children aged 15 or younger in New South Wales	Australia	Myringotomy, tonsillectomy, adenoidectomy, or tonsillectomy and adenoidectomy	Low	1993 NSW Health Department Working Party: Guidelines on the management of pediatric middle ear disease	Procedure rates decreased immediately after guideline introduction, but subsequently increased in the next few years. Overall rate of procedures increased by 21% during the study period.	No
Sherman et al44 1992	RC	1983–1989	56,406 patients SEER and Medicare databases	United States	Radical prostatectomy	High	1987 NIH CDC on management of clinically localized prostate cancer	Preconference procedure rate was increasing at 2% per year and did not change after the consensus conference.	No
Studies evaluating effects on the choice of procedure
Du et al45 2000	RC	1983–1995	169,466 patients SEER database	United States	Breast conservation therapy	High	1990 NIH CDC on treatment of early stage breast cancer	Overall BCT rates increased from 27.5% to 42.6% after guideline release. The rate of BCT use increased by 2.37% in the initial period after the conference.	Yes
Fukada et al46 2009	RC	1996–2007	2199 patients 5 hospitals	Japan	Breast conservation therapy	High	1999 Japanese Breast Cancer Society Practice Guideline: Breast Conserving Surgery	The proportion of patients undergoing BCT increased from 26.4% to 59.9% after guideline publication.	Yes
Kosecoff et al51 1987	RC	1977–1981	2770 patients 10 hospitals	United States	Total mastectomy with axillary dissection	Low	1979 NIH CDC on treatment of primary breast cancer: management of local disease	The proportion of patients receiving total mastectomy with axillary dissection increased from 74% in 1977–1978 to 84% after guideline publication in 1980–1981.	No
Lazovich et al47 1997	RC	1983–1993	13,551 patients SEER database	United States	Breast conservation therapy	High	1990 NIH CDC on treatment of early stage breast cancer	In this study, BCT rates increased from 36.9% in 1983–1985 to 54.9% in 1990–1993 for stage I patients and from 25.1% to 35.2% in stage II patients.	Yes
Lazovich et al48 1999	RC	1983–1995	109,880 patients SEER database	United States	Breast conservation therapy	High	1990 NIH CDC on treatment of early stage breast cancer	In this study, BCT rates increased from 34.6% in 1985–1990 to 53.4% in 1990–1995. When compared to 1983–1985, the adjusted OR of BCT in 1990–1995 was 3.39 for stage I and 3.11 for stage II patients.	Yes
Nattinger et al49 2000	RC	1983–1995	145,490 Patients SEER database	United States	Breast conservation therapy	High	1990 NIH CDC on treatment of early stage breast cancer	After guideline release, BCT rates increased between 1990 and 1995 from approximately 30% to 50%.	Not reported
Struikmans et al50 2011	RC	1997–2008	65,966 Patients 4 cancer centers	The Netherlands	Breast conservation surgery and radiotherapy	High	2002 Working Group Treatment Breast Cancer: Guideline Treatment Breast Cancer	The proportion of patients receiving breast conservation surgery increased from 32% to 45% in 1997 to 41% to 57% in 2008, whereas use of mastectomy decreased from 50% to 61% to 36% to 53%. Use of postoperative radiotherapy increased in all regions.	Not reported
Studies evaluating effects on regional variation
Brownell42 2002								Regional variation was present in all study areas and changed over the study period. In 1998–1999, non-Winnipeg children were 30.5% more likely to undergo tonsillectomies than Winnipeg children and tonsillectomy rates varied from 2.3/1000 to 6.9/1000 across regions.	No
Lazovich et al47 1997								After 1990, the rates of BCT rose from 41.6% to 50.8% in King County and from 19.4% to 35.8% in nearby areas. Overall, there was a decrease in regional variation over the study period.	Yes
Lazovich et al48 1999								Rates of BCT and regional variation increased in all 9 SEER regions during the study period. Before 1990, rates of BCT was in the range from 12.6% to 33.9% and after 1990, from 26.7 to 55.6%.	No
Mason et al41 2001								In 1992, regional and national per capita procedure rates varied by a factor of 2. This variation decreased by 30% after bulletin dissemination.	Yes
Struikmans et al50 2011								Statistically significant differences in utilization of postoperative radiotherapy between centers existed in 1997, but not in 2008.	Yes

TABLE 2.

Summary of Studies Evaluating Shared Decision Making and Decision Aids

Author/s, Year	Study Design	Years Studied	Study Population	Location	Surgery Evaluated	Quality	Summary of Findings	Statistically Significant Effect
Studies evaluating effects on population-based rates of surgery
Arterburn et al15 2012	Prospective cohort	2009–2010	1310 patients 27 surgeons 5 sites	United States	Total hip replacement, total knee replacement	High	The introduction of decision aids led to 26% fewer total hip replacements and 38% fewer total knee replacements.	Yes
Auvinen et al52 2004	Randomized controlled trial	1993–1994	210 patients 4 hospitals	Finland	Radical prostatectomy or orchiectomy	High	In this study, 58% of patients in the enhanced participation arm opted for radical prostatectomy or orchiectomy, whereas 86% of patients in the standard practice arm opted for surgical therapy.	Yes
Bernstein et al55 1998	Randomized controlled trial	1996–1997	217 patients 2 institutions	United States	CABG	High	In this study, 21% of patients using a decision aid chose CABG vs 38% of patients in the usual care group.	No
Deyo et al54 2000	Randomized controlled trial	Not given	393 patients 2 institutions	United States	Lumbar spine surgery	High	The overall surgery rate was 22% lower for patients receiving the videodisc. Patients with herniated disks underwent significantly less surgery, whereas spinal stenosis patients chose surgery more often.	Mixed
Kennedy et al53 2002	Randomized controlled trial	1996–1998	894 patients 6 hospitals	United Kingdom	Hysterectomy	High	In this study, 38.2% of patients receiving the decision aid and interview, 48% of those receiving standard practice, and 48% of those receiving the decision aid alone chose hysterectomy.	Yes
Protheroe et al58 2007	Randomized controlled trial	2003–2005	146 patients 200 group practices	United Kingdom	Hysterectomy or transcervical endometrial resection	High	In this study, 13% of patients receiving a decision aid and informational leaflet chose surgical treatment compared with 5% of patients receiving an informational leaflet alone.	No
Schwartz et al56 2009	Randomized controlled trial	2001–2005	214 patients 2 institutions	United States	Risk reducing mastectomy	Low	In this study, 18% of patients using a decision aid selected risk reducing mastectomy compared with 13% of patients receiving usual care.	No
Tiller et al57 2006	Randomized controlled trial	Not reported	131 patients 6 familial cancer clinics	Australia	Prophylactic oopherectomy	Low	In this study, 34% of women with a family history of breast cancer, ovarian cancer, or HNPCC chose prophylactic oophorectomy after viewing a decision aid compared with 30.3% of patients that received only the informational pamphlet.	No
Vuorma et al59 2004	Randomized controlled trial and prospective cohort	1997–1999	363 patients 14 hospitals	Finland	Hysterectomy, uterus saving surgery	High	In this study, 51% of patients receiving the decision aid, 48% of patients receiving usual care, and 53% of those in the pretrial group chose hysterectomy, whereas 10% of patients receiving the decision aid, 16% of patients receiving usual care, and 6% of those in the pretrial group chose uterus saving surgery	No
Wagner et al65 1995	Retrospective cohort and ecological	1989–1991	451 patients 2 KP regions	United States	Transurethral resection of the prostate	High	In Colorado patients receiving the shared decision-making program, rates of TURP decreased from 6 to 2.5 (per 1000) between 1988 and 1991. In Washington patients receiving the same program, rates fell from 4.9 to 2.3 between 1987 and 1990, but increased in 1991.	Mixed
Studies evaluating effects on the choice of procedure
Molenaar et al60 2001	Prospective cohort	1996–1998	180 patients 3 hospitals	The Netherlands	Breast conservation therapy	Low	In this study, patients who used a decision aid chose BCT in 75% of cases as compared with 68% of those receiving standard care.	No
Street et al61 1995	Randomized controlled trial	1993–1994	60 patients 1 hospital	United States	Breast conservation therapy	Low	In this study, patients who used a decision aid chose BCT in 76% of cases as compared with 58% of those viewing a brochure only.	No
Vodermaier et al62 2009	Randomized controlled trial	2003–2004	152 patients 1 institution	Germany	Breast conservation therapy	High	In this study, patients who used a decision aid chose BCT in 94.9% of cases as compared with 87.8% of those receiving standard care.	No
Whelan et al65 1999	Prospective cohort	1996–1997	369 patients 7 surgeons	Canada	Breast conservation therapy	High	The rate of BCT was 88% before use of a decision board during surgical consultation and decreased to 73% after its introduction.	Yes
Whelan et al64 2004	Randomized controlled trial	1999–2002	201 patients 20 surgeons	Canada	Breast conservation therapy	High	Patients viewing a decision board during surgical consultation chose BCT 94% of the time as compared with 76% of women receiving usual care.	Yes
Studies evaluating effects on regional variation
Wagner et al65 1995							In 1991, the TURP rate in KP Colorado was 60% lower than other CO regions. The KP region in Washington performed 50% fewer TURPs in 1990 than other WA regions, but this reduction did not persist the following year.	Unclear

CABG indicates coronary artery bypass graft; HNPCC, hereditary nonpolyposis colorectal cancer.

Most studies were published within the last 20 years. Twelve (44%) studies originated from the United States and 15 (56%) were international. The majority of articles examining guidelines were observational studies that utilized a retrospective cohort time-series design, whereas most studies of decision aids were randomized-controlled trials. Breast surgery was the most common surgery evaluated (48%), but the type of surgery examined varied significantly between studies. Most studies were considered of high quality (19 of 27, 70%). The Harbord test evaluating for possible small-study effects did not achieve significance, and funnel plot did not reveal significant asymmetry. Measures of the influence of practice guidelines or decision aids on the use of surgery can be characterized in 3 ways: effects on population-based rates of surgery, effects on the choice of procedure, and effects on regional variation in the use of surgery.

Practice Guidelines and Consensus Statements

The details of studies examining the influence of clinical practice guidelines or consensus statements on rates of surgery use are listed in Table 1. Of the 12 studies, 7 evaluate breast cancer surgery, 4 evaluate pediatric ear, nose, and throat surgery, and 1 study evaluates radical prostatectomy.

Population-Based Rates

Five studies evaluated population-based rates of procedures to examine the influence of practice guidelines on the decision to perform surgery. Although 4 of 5 studies found a measurable change in rates reflecting guideline recommendations, only 2 studies report a statistically significant impact. Both of these studies evaluate the 1992 NHS Effective Health Care bulletin’s influence on pediatric tympanostomy tube rates in NHS hospitals.^40,41 Of the remaining 3 studies, 2 evaluating pediatric tonsillectomy found rates decreased immediately after guideline dissemination, but returned to baseline or higher levels in the following years.^42,43 The single study of radical prostatectomy rates for prostate cancer found that annual rates steadily increased (1%–2% per year) during the study period, but determined this was not attributable to guideline release.⁴⁴

Procedure Choice

Seven studies evaluated the particular choice of surgical procedure in the treatment of breast cancer. Most of these studies reported an increase in breast conservation therapy (BCT) after implementation of clinical practice guidelines that encouraged its use in certain populations. Four of these studies found the increase in BCT was statistically significant,^45–48 whereas 2 studies reported sizable increases in rates (ranging from 10% to 20%) but did not comment on statistical significance.^49,50 The only study that did not find a statistically significant effect examined the use of total mastectomy with axillary dissection after a National Institute of Health Consensus Development Conference on primary breast cancer in 1979.⁵¹ Despite not achieving significance, it reported an increase in rates of recommended therapy by 10% during the period.

Regional Variation

Five studies of guideline dissemination included an analysis of changes in regional variation. The study by Mason et al⁴¹ evaluating tympanostomy tube insertion in NHS hospitals found that regional variation decreased by 30% following the NHS bulletin. Although not evaluating surgery directly, Struikmans et al⁵⁰ reported that the variation between Dutch centers in radiotherapy utilization after breast conserving surgery was significant before, but not after, the release of multidisciplinary breast cancer treatment guidelines in 2002. Lazovich et al⁴⁷ found that the range of BCT rates in 3 counties in Washington state was narrowed after the 1990 NIH Consensus Development Conference (19.4%–41.6% before, 35.8%–50.8% after). However, a subsequent study of national data found that the range of BCT rates widened in 9 SEER (Surveillance, Epidemiology, and End Results) registry regions (12.6%–33.9% before, 26.7%–55.6% after).⁴⁸ Brownell⁴² reported that substantial regional variation in the rate of tonsillectomies in Manitoba persisted throughout the period studied.

Decision Aids and Shared Decision-Making Tools

Studies evaluating the impact of decision aids and shared decision-making programs in clinical practice are listed in Table 2. The type of procedure examined varied substantially and included surgery for breast cancer, menorrhagia, familial cancer syndromes, benign prostatic hypertrophy, prostate cancer, back pain, ischemic heart disease, and osteoarthritis.

Population-Based Rates

Ten of the included studies examined the influence of decision aids on population-based rates of surgery. Although 3 of these studies reported a statistically significant effect, the direction of influence varied according to the specific operation and clinical context. The strongest impact of decision aids was reported by Arterburn et al¹⁵ in a prospective study of osteoarthritis patients who viewed a multimedia decision aid before surgical consultation. In this group of patients, rates of total knee replacement decreased by 38%, and rates of total hip replacement decreased by 26% during the 6-month period. Auvinen et al⁵² reported a similar decrease in a study evaluating the impact of an enhanced participation program on rates of surgery for prostate cancer. The authors found that only 58% of enhanced participation patients chose surgical therapy, compared with 86% of standard practice patients. A third trial reporting a significant effect compared standard practice, a decision aid, and a decision aid with a preference elicitation interview in women with menorrhagia. This study found that addition of the interview decreased the rate of hysterectomies performed by 10%.⁵³

The study by Deyo et al⁵⁴ found that although the relative difference of 22% in overall rates of lumbar spine surgery was not significant, the subgroup of patients with herniated discs underwent significantly less surgery throughout the year (32% vs 47% in the booklet group). No significant effects were found after use of decision aids for coronary revascularization,⁵⁵ risk reducing mastectomy,⁵⁶ prophylactic oophorectomy,⁵⁷ or gynecologic procedures for menorrhagia.^58,59

Procedure Choice

Five studies investigated the rate of a particular procedure chosen after the need for surgery was established in breast cancer treatment. Three of these did not report a statistically significant change in the rate of a specific procedure after decision aid administration.^60–62 The 2 studies that demonstrated significant effects were both performed by Whelan and colleagues and evaluated the rate of BCT. While the study from 1999 found that rates of BCT use decreased from 88% to 73% following administration of a decision board by surgeons,⁶³ the 2004 study reported an increase in BCT rates from 76% to 94%.⁶⁴

Regional Variation

Wagner et al⁶⁵ performed the only study to evaluate the influence of decision aids on regional variation. This retrospective study evaluated the impact of a videodisc-based shared decision-making program for Kaiser-Permanente (KP) patients with benign prostatic hypertrophy. The authors compared KP regions in Washington and Colorado that utilized the decision aid with surrounding nonstudy KP regions. After implementation of the decision aid, the investigators found that the 1990 rate of transurethral resection of the prostate (TURP) in the Washington KP region was 50% that of other Washington regions (95% confidence interval, 33%–77%), and the 1991 TURP rate in KP Colorado was 60% lower than other Colorado regions (95% confidence interval, 47–74). However, although this decrease persisted in Colorado, it did not persist in Washington at the conclusion of the study.

DISCUSSION

This is the first systematic review to focus on the impact of clinical practice guidelines and decision aids on population-based rates of surgery, patient procedure choice, and regional variation. Despite heterogeneous patient populations, study designs, and measures of effect among the examined studies, this review highlights the potential of both strategies to influence use of surgery and regional variation.

Half of the studies evaluating the influence of practice guidelines on use of surgery reported statistically significant effects. All studies examining the effect of guidelines on procedure choice (ie, the selection of one procedure over another) found a measurable increase in use of the recommended procedure. Furthermore, of the studies that examined impact on regional variation, 3 of 5 described a significant reduction after guideline implementation. On the contrary, studies that described the impact of guidelines on overall population-based rates were mixed. Although procedure rates in most studies showed some impact, this effect was often transient. In many cases, population-based rates later returned to the prestudy rate, negating the effects of the initial intervention.

To a large degree, our findings mirror the conclusions of previous work examining the influence of guidelines. Grimshaw and colleagues^66,67 found that explicit guidelines can be highly effective in improving patient care when supported by rigorous evaluations. A more recent Cochrane review concluded that clinical practice guidelines and other printed materials may have a beneficial effect on practice, but the review was unable to estimate the effect on outcomes or the observed effect size.⁶⁸ Given that guidelines are heavily dependent on many factors like clinical context and methods of development, dissemination, and implementation,^69–71 it comes as no surprise that the impact of individual guidelines on clinical practice may vary widely, as suggested by our results. However, in many clinical contexts, our findings confirm that guidelines have the potential to influence use of surgery and reduce the wide variation observed across geographic areas.

We found similar results in our analysis of studies evaluating the impact of decision aids. Although only one study evaluated their effects on regional variation, with mixed results, statistically significant effects were observed in one third of the included studies. The introduction of decision aids led to reductions in population-based procedure rates in many studies examined. However, 1 study of menorrhagia patients, and 2 studies of prophylactic surgery for patients at high-risk for cancer, reported small but nonsignificant increases in procedure rates. Almost all studies evaluating a patient’s choice of procedure found that decision aids increased use of the less invasive option. Although these findings highlight the potential of decision aids to influence use of procedures and possibly reduce variation, these results also illustrate that the direction of influence is highly dependent on the specific circumstance and clinical context.

Although this is the first review to specifically evaluate the influence of decision aids on population-based rates of surgery, multiple recent reviews corroborate these findings. A 2011 review from the Cochrane Collaboration confirmed that the use of decision aids enhances patient decision making across multiple other important domains.²⁰ The authors found that decision aids increase knowledge scores, improve patient perceptions of risk, lessen decisional conflict, decrease the proportion of patients who assume a passive role and who remain undecided, and increase consistency between patients’ informed values and the treatment chosen. Although not a primary outcome of their analysis, they also reported decreases in the utilization of major surgery with use of some decision aids. A more recent review by Knops and colleagues⁷² also reported that patients using decision aids chose less invasive treatments more frequently.

This systematic review has several limitations. First, because there was no single term within the controlled vocabularies (eg, Medical Subject Headings, EMTREE) to reflect the rate of surgery concept, broad statistical subheadings, and controlled terms were used in combination with title and abstract keywords. Although such an approach is sufficiently sensitive for this review, it could have resulted in a failure to identify some articles that meet inclusion criteria. To ensure this risk was minimized, the keyword search was kept broad and articles with uncertain eligibility were included for further review at each stage. In addition, the database search was supported by multiple searches of common search engines and a comprehensive backward and forward bibliography search of included studies. A second limitation is the substantial heterogeneity of published studies with regards to study design, patient populations, and outcome measures, which led to considerable variability within the results of individual studies and precluded formal meta-analysis.

Third, most of the studies evaluating guidelines utilized a retrospective time-series observational study design. Although replicate measures may help identify regression to the mean, this design cannot separate effects of the intervention in question from secular trends. And fourth, many of the included studies report on a small population size and are likely inadequately powered to detect a statistically significant effect. Despite this potential bias toward the null hypothesis, this review was able to identify evidence to support the use of practice guidelines and decision aids to reduce the regional variation observed in use of surgery.

The current literature evaluating the impact of guidelines and decision aids on use of surgery and regional variation is deep but limited in clinical scope. Of the 27 studies included in this review, 75% focus on 4 surgical conditions: breast cancer, prostatic disease, tonsillitis, and recurrent ear infection. As a result, the generalizability of these studies toward other clinical contexts remains uncertain, and suggests the need for further research. This gap in knowledge presents a significant opportunity for health care researchers. Professional and specialty organizations, in turn, should continue to encourage, develop, and disseminate new guidelines and decision tools using the highest-quality evidence available, to maximize the potential effectiveness of these strategies.

Although the primary focus of this review was to assess the comparative effectiveness of guideline dissemination and decision aids in reducing variation, the broader dissemination of these tools into real world clinical practice will depend on additional factors. Incentives for physicians to use them would no doubt accelerate adoption. Accountable care organizations, shared savings programs, and risk-based reimbursement (including capitation) may better establish the “business case” for reducing regional variation in the use of surgery.⁶ Payers could create additional incentives through pay-for-performance programs.¹⁸

In addition to financial incentives, the use of practice guidelines and decision aids could be enhanced by reducing practical barriers to their implementation in everyday clinical practice. As laid out in the PARIHS framework, for example, successful implementation of clinical interventions depends on not only high-quality evidence, but also a receptive environment, facilitation, and support.^73,74 Buy-in from surgeons will be essential in establishing that environment. Rather than view guidelines and decision aids as threats to their professional autonomy, surgeons could be proactive and take the lead in the development of these tools and the processes by which they are incorporated into day-to-day clinical decision making. Advances in electronic health record systems and information technology may also help minimize clinical workflow disruptions and accelerate adoption.

CONCLUSIONS

Although current implementation barriers should not be underestimated, findings from this review suggest that practice guidelines and decision aids could help surgeons improve clinical decision making for individual patients and populations with many clinical conditions.