A Systematic Review and Meta-Analysis of Health Care Utilization Outcomes in Alcohol Screening and Brief Intervention Trials

Abstract

Objective

This systematic review and meta-analysis examines the effect of screening and brief intervention (SBI) on outpatient, emergency department (ED), and inpatient health care utilization outcomes. Much of the current literature speculates that SBI provides cost savings through reduced health care utilization, but no systematic review or meta-analysis examines this assertion.

Method

Publications were abstracted from online journal collections and targeted Web searches. The systematic review included any publications that examined the association between SBI and health care utilization. Each publication was rated independently by two study authors and assigned a consensus methodological score. The meta-analysis focused on those studies examined in the systematic review, but it excluded publications that had incomplete data, low methodological quality, or a cluster randomized design.

Results

Systematic review results suggest that SBI has little to no effect on inpatient or outpatient health care utilization, but it may have a small, negative effect on ED utilization. A random effects meta-analysis using the Hedges method confirms the ED result for SBI delivered across settings (SMD = −.06, I-squared = 13.9%) but does not achieve statistical significance (CI: −0.15, 0.03).

Conclusions

SBI may reduce overall health care costs, but more studies are needed. Current evidence is inconclusive for SBI delivered in ED and non-ED hospital settings. Future studies of SBI and health care utilization should report the estimated effects and variance, regardless of the effect size or statistical significance.

Introduction

A number of systematic reviews and meta-analyses assess the effectiveness of screening and brief intervention (SBI) to reduce alcohol consumption among at-risk drinkers.^1-12 Despite concerns about some studies' methodology,^5,9 SBI is widely thought to reduce alcohol consumption. Beyond clinical outcomes, SBI is also considered to be cost-effective or cost-beneficial by many authors.^11,13-16 Furthermore, many U.S. policy makers—such as the Substance Abuse and Mental Health Services Administration¹⁷—advocate for the widespread adoption of SBI, often stating that SBI will reduce health care utilization and therefore save money.

Several recent studies and reviews of the economic evaluation of SBI have been published,^13-16 but to date there has been no systematic review and meta-analysis that assesses whether SBI reduces health care utilization. A recent review¹⁶ suggests SBI can be cost-effective in improving quality-adjusted life years (QALYs) when implemented in primary care settings. However, evidence of a cost-effective intervention does not necessarily imply reduced health care utilization or health care cost savings. This article presents a systematic review and meta-analysis of the effect of SBI on health care utilization to evaluate the gap between the literature and broad policy support for SBI.

Methods

Systematic Review

A systematic literature search and review was conducted using electronic databases, formal selection criteria, and multiple reviewers. A literature search was conducted using several databases: CINAHL, MEDLINE, PubMed, Cochrane, EBSCO, JSTOR, and PsycARTICLES. Search terms comprised combinations of brief intervention terminology (screening and brief intervention, alcohol brief intervention, brief intervention, SBI, BI, and alcohol) and health care outcomes (health, health care utilization, utilization, physician visit, emergency department visit, general practitioner visit, hospital stays, hospitalization, hospital readmission, cost, and cost-effectiveness). All identified publications' reference lists, including those from other systematic reviews or meta-analyses on SBI, were also used.

Searches were not limited by year of publication (dates ranged from 1962 to 2010), but publications unavailable in English were excluded. The primary inclusion criterion was a health care utilization outcome in an alcohol-focused publication. Publications were then reviewed using the following 3 criteria: (1) conducted a form of brief intervention, (2) involved a solely non-alcohol-dependent population, and (3) was an independent publication (e.g., not a review or meta-analysis). Publications targeting alcohol-dependent populations were excluded.

The relationship between SBI and health care utilization may depend on both the setting in which SBI is delivered (e.g., primary care vs. ED) and the type of health care utilization (e.g., inpatient stay vs. outpatient visit). Publications were thus categorized into medical settings based on where SBI was delivered: primary care, ED, and non-ED hospital. As noted by Kraemer¹⁵ and Kaner et al.,⁷ the quality and outcome of SBI delivered in each of these 3 settings differ greatly.

Health care utilization is classified into outpatient care, ED care, and inpatient care. The types of care assigned for study outcomes were based on the descriptions in each publication. Outpatient care includes visits to a primary care provider/general practitioner, nurse practitioner, or outpatient counselor. In many cases, the outpatient care category is a catch-all for services not otherwise classified, such as ambulatory hospital or laboratory.¹⁸ ED care includes any hospital ED or urgent or trauma care facility visit. Inpatient care includes any non-ED hospital stay or admission or inpatient treatment facility stay. Setting is not tied specifically to the type of outcome. For example, a study conducted in a primary care setting may examine inpatient care.

The literature search and review was conducted by a study author. Two other authors conducted targeted, random sample screenings to ensure quality and accuracy. Several publications in this review use the same study for source data: Project TrEAT,^19-24 Project GOAL,^25,26 the Radcliffe Study (Radcliffe Hospital, Oxford, United Kingdom),^27,28 and the St. Mary's Hospital study (St. Mary's Hospital, London, United Kingdom).^29,30 So that the results of this analysis are not disproportionately influenced by any 1 study, the analyses include only those publications from any 1 study with the most comprehensive set of outcomes, the longest follow-up period, and/or exclusive target populations. For Project TrEAT, 3 of 6 publications were included: Fleming et al.,²¹ Grossberg et al.,²³ and Manwell et al.²² Fleming et al.²¹ is the most comprehensive set of main findings. Grossberg et al.²³ and Manwell et al.²² represent specific sub-analyses on young adults and women, respectively. Barrett et al.³⁰ was selected for the St. Mary's study over Crawford et al.²⁹ for a more comprehensive analysis. Mundt et al.²⁶ was selected to represent Project GOAL because that publication had the longest follow-up period of the available project publications. For the Radcliffe Study, Anderson and Scott²⁸ and Scott and Anderson²⁷ were selected because those publications used mutually exclusive male and female cohorts.

A qualitative methodological score was assigned to each publication in the meta-analysis. The method followed was that of Miller et al.,³¹ as described in Vasilaki et al.,¹⁰ Miller and Willbourne,³² and Bien et al.,³ using a 12-item assessment of methodological quality and design. Summary scores range from 0 to 17, with 14 out of 17 indicating an excellent methodological quality.³² Each article was scored independently by 2 of the contributing authors. Any disagreement on scoring was resolved by the authors to obtain a consensus score.

Meta-Analysis

Data for the meta-analysis were abstracted from each publication by 1 author and reviewed by another author for accuracy. For publications that did not contain the necessary statistical components for the health care outcomes (sample size, effect size, variance measure), the corresponding author was contacted. Authors of publications published before 1995 were not contacted because of the anticipated infeasibility of retrieving the data or estimates. If the corresponding author could not provide the requested information, the publication was excluded from the meta-analysis. If the corresponding author provided data files instead of summary statistics, Stata 11 was used to calculate continuous effect sizes and standard deviations.

A random-effects specification with the Hedges method was used for the meta-analysis using the “metan” command in Stata 11. A random-effects model was selected because publications vary substantially in setting, form, and quality of the intervention and in the definition of the health care utilization outcomes. Publications were excluded from the final meta-analysis if data were not available, the publication was of poor methodological quality, or the publication used a cluster randomized design. The publications with cluster randomized designs did not provide enough information to include clustering effects in the meta-analysis appropriately.

The standardized mean difference (SMD) and associated 95% confidence interval were assessed for each publication and pooled for all publications. An SMD of 0.20 or less is considered small.³³ The I-squared statistic is included to interpret the heterogeneity around the pooled SMD. Heterogeneity is categorized into 3 levels: low (I² = ∼20%), moderate (I² = ∼50%), and high (I² = ∼70%). Higher levels of heterogeneity indicate greater variability across publications, in which case the pooled SMD may not be representative of the publications in the analysis. Given the aforementioned variation in setting, form, and quality of the intervention, moderate to high heterogeneity should be expected. This analysis examined 1 publication per study (the “main findings” publication). An alternative analysis included additional publications on Project TrEAT and the St. Mary's study.

Forest plots for the meta-analysis are presented separately by type of care. Although it is preferred to also present forest plots separately by setting, there were insufficient publications to review each setting separately.

Results

Systematic Review

For the systematic review, 216 publications were identified and abstracted for further review. Of these, 56 contained a health care utilization outcome and met the basic inclusion criteria; 29 publications met the full list of inclusion criteria and were selected for review. Table 1 describes the key characteristics of the publications by type of care and setting. Within setting, publications are presented by country, author, and year. Twenty-one publications were conducted in a primary care setting,^18-28,34-43 Four were conducted in an ED setting,^29,30,44,45 and 4 were conducted in a hospital setting other than an ED.^46-49 Seventeen of the 29 publications were set in the United States, 6 in the United Kingdom, 2 in Australia and Sweden, and 1 in Canada and Switzerland. Table 1 also presents the qualitative methodological scoring of each publication. Scores ranged from 5 to 16; the mean score was 13.17, and the median was 13.

Table 1. Characteristics of Reviewed Studies.

Author (Year)	Country	Popu- lation	Follow-Up Period	Sample Size (Total/Int/Con)	Study Design	QMS	Outpatient Measure	Direction	Significant Effect?	ED Measure	Direction	Significant Effect?	Inpatient Measure	Direction	Significant Effect?
Primary Care Setting
Wutzke et al,.200234	Australia	General	10 year	554/410/144	RCT	13	Outpatient visits	Increase	No sig. effect	NA	NA		Use of health care facilities	Increase	No sig. effect
Israel et al., 199635	Canada	General	12	73/38/35	RCT	15	GP visits	Decrease	Sig. effect	NA	NA		NA	NA	-
Kristenson et al., 198336	Sweden	General	24, 60	414/219/195	Mixed QE/ RCT	10	NA	NA		NA	NA		Hospital days	Decrease	Not tested
Tomson et al. 199837	Sweden	General	36	75/30/45	RCT	13	Consultations (mean)	No change	No sig. effect	NA	NA		Admission rate (%)	Increase	No
Heather et al., 198738	UK	General	6	91/59/32	RCT	14	GP consultations	Increase	No sig. effect	NA	NA		NA	NA
Lock et al., 200639	UK	General	12	127/67/60	Cluster RCT	11	GP visits (mean), NP visits (mean)	Decrease	No sig. effect	ED visits (mean)	Decrease	No sig. effect	Hospital inpatient care	Increase	No
Anderson and Scott, 199228	UK	Male	12	154/80/74	RCT	13	Consultation Rate (mean)	Increase	No sig. effect	NA	NA		Episode Rate (mean)	Increase	No sig. effect
Fleming et al., 200440	US	Diabetic, hyperten-sive, other medical	12	13570/65	RCT	16	NA	NA		ED visit in last 12 months, urgent care visit in last 12 months	Not given	No sig. effect	Hospitali-zation in last 12 months	Not given	No sig. effect
Fleming et al., 199719	US	General	6, 12	774/392/382	RCT	16	NA	NA		ED visits in last 6 months	Decrease	No sig. effect	Hospital days	Decrease	No sig. effect
Senft et al., 199741	US	General	12	514/260/254	RCT	15	Outpatient visits (mean)	Increase	No sig. effect	NA	NA		Hospitali-zation rate	Increase	No sig. effect
Fleming et al., 200020	US	General	6, 12	774/392/382	RCT	16	NA	NA		ED visits in last 6 months	Decrease	No sig. effect	Hospital days	Decrease	Sig. effect
Fleming et al., 200221	US	General	48	774/392/382	RCT	16	NA	NA		ED visits in last 6 months	Decrease	Sig. effect	Hospital days	Decrease	Sig. effect
Freeborn et al., 200042	US	General	24	514/260/254	RCT	15	Outpatient visits (mean, if any)	Decrease	No sig. effect	NA	NA		Mean Hospital days (if any)	Decrease	No sig. effect
Mundt et al., 200624	US	General	12	774/392/382	RCT	16	NA	NA		Total no. ED visits	Decrease	Not tested	Total no. days hospitalized	Decrease	Not tested
Bray et al., 200718	US	General	12	3628/1945/1683	Cluster RCT	12	Outpatient visits (mean),	Mixed	No sig. effect	ED visits (mean)	Mixed	No sig. effect	Inpatient days (mean), ADM days (mean)	Mixed	No sig. effect
Fleming et al., 199925	US	Older Adults	6, 12	158/87/71	RCT	15	NA	NA		ED visits in last 6 months	Not given	No sig. effect	Hospital days	Not given	No sig. effect
Mundt et al., 200526	US	Older Adults	24	158/87/71	RCT	15	NA	NA		ED visits in last 6 months	Not given	No sig. effect	Hospital days	Not given	No sig. effect
Copeland et al., 200343	US	Veteran	9, 18	205/100/105	RCT	12	Total outpatient stops (mean)	Increase	sig. effect	NA	NA		Total inpatient stops (mean)	Increase	No sig. effect
Scott and Anderson, 199027	US	Women	12	72/33/39	RCT	12	Consultation rate (mean)	Decrease	No sig. effect	NA	NA		Episode Rate (mean)	Decrease	No sig. effect
Manwell et al., 200022	US	Women	48	205/103/102	RCT	16	NA	NA		ED visits in last 6 months	Decrease	No sig. effect	Hospital days	Decrease	No sig. effect
Grossberg et al., 200423	US	Young Adults	48	226/114/112	RCT	16	NA	NA		ED visits	Decrease	Sig. effect	Hospital days	Decrease	No sig. effect
ED Setting
Daeppen et al., 200744	Switzer-land	Injured	12	770/236/534	RCT	12	Medical consultations (mean)	Increase	No sig. effect	NA	NA		Hospital days (mean)	Increase	No sig. effect
Crawford et al., 200429	UK	General	12	377/182/195	RCT	13	NA	NA		ED visits (mean)	Decrease	Sig. effect	NA	NA	-
Barrett et al., 200630	UK	General	12	290/131/159	RCT	13	Outpatient hospital visit (mean), GP contacts (mean)	Increase	Not tested	ED visits (mean),	Decrease	Not tested	Inpatient days (mean)	Decrease	Not tested
Gentilello et al., 199945	US	General	12	409/194/215	RCT	14	NA	NA	-	Risk of repeat injury requiring ED visit (%)	Decrease	Sig. effect	NA	NA	-
Non-ED Hospital Setting
Shourie et al., 200646	Australia	Overnight Surgery Patients	6	106/45/61	QE	8	GP visits	No change	No sig. effect	NA	NA		Hospital readmission rate (%)	No change	No sig. effect
Watson et al., 199947	UK	General	12	102/71/31	QE	7	GP visits	Not given	No sig. effect	NA	NA		Hospital visits	Not given	No sig. effect
Saitz et al., 200748	US	Inpatient	12	287/141/146	RCT	13	NA	NA		ED visits	Not given	No sig. effect	Hospital days	Not given	No sig. effect
Storer, 200349	US	Military	12	444/206/238	Obser-vation	5	NA	NA	-	NA	NA		Hospital days (mean), hospital readmission rate (%)	Not given, Decrease	Not tested, sig. effect

Note: Int = intervention, Con = control, ADM = alcohol, drug, or mental health, ED = emergency department, GP = general practitioner, NP = nurse practitioner, RCT = randomized control trial, QE = quasi-experimental

Table 2 summarizes the findings by type of care and setting. In the primary care setting, 11 of the 21 publications measured outpatient care. The evidence appears to be evenly split between decreased and increased utilization, suggesting no real effect. One publication³⁵ found a statistically significant decrease in outpatient utilization, and 3 others^27,39,42 reported decreases that were not statistically significant. One publication⁴³ found a statistically significant increase in outpatient utilization, and 4 reported increases that were not statistically significant.^28,34,38,41 Tomson et al.³⁷ found no differences at follow-up.

Table 2. Summary of Health Care Utilization Outcomes Identified in the Systematic Review.

	Primary Care Setting	ED Setting	Non-ED Hospital Setting
Outpatient utilization measured	11 of 21	2 of 4	2 of 4
Significant decrease	1	-	-
Non-significant decrease	3	-	-
Significant increase	1	-	-
Non-significant increase	4	2	-
No effect or mixed effect	2	-	2
ED utilization measured	11 of 21	3 of 4	1 of 4
Significant decrease	2	2	-
Non-significant decrease	5	1	-
Significant increase	-	-	-
Non-significant increase	-	-	-
No effect or mixed effect	4	-	1
Inpatient utilization measured	19 of 21	2 of 4	4 of 4
Significant decrease	2	-	1
Non-significant decrease	7	1	-
Significant increases	-	-	-
Non-significant increases	6	1	-
No effect or mixed effect	4	-	3

Note: ED = emergency department

For ED care, the evidence in the primary care setting indicates a statistically insignificant decrease in ED utilization (11 of 21 possible). Two TrEAT publications^21,23 reported a statistically significant decrease. The remaining Project TrEAT publications^19,20,22,24 and 1 independent publication³⁹ reported decreases that were statistically insignificant.

For inpatient care, the evidence in the primary care setting (19 of 21 possible) suggests no real effect, with little consensus on the direction or magnitude of any potential effect. Two Project TrEAT publications reported statistically significant decreases,^20,21 and 7 other publications reported decreases that were statistically insignificant; 4 use TrEAT data,^19,22-24 and 3 are independent.^27,36,42 Six publications^{28,34,37,39,41,43} reported increases in inpatient utilization that were statistically insignificant.

Across all 3 types of care in the primary care setting, several publications reported mixed results or no effect. Bray et al.¹⁸ reported mixed, insignificant results. Project GOAL^25,26 and 1 independent publication⁴⁰ indicated no differences.

Findings for health care utilization in the ED and non-ED hospital settings were inconclusive largely due to an insufficient number of publications. There were no statistically significant findings for outpatient care in the ED setting; 2 publications^30,44 found an insignificant increase in outpatient care. Both St. Mary's study publications found reduced ED care; Crawford et al.²⁹ was statistically significant, but Barrett et al.³⁰ was statistically insignificant. For inpatient care, there was 1 statistically significant decrease,⁴⁵ 1 statistically insignificant decrease,³⁰ and 1 statistically insignificant increase.⁴⁴

In the non-ED hospital setting, 1 statistically significant decrease was found for inpatient care.⁴⁹ Three other publications found mixed or no effects for outpatient and ED care.^46-48

Meta-Analysis

The following publications were excluded from the meta-analysis because complete data were not available: Israel et al.,³⁵ Fleming et al.,⁴⁰ Kristenson et al.,³⁶ Tomson et al.,³⁷ and Freeborn et al.⁴⁶ Several publications were also excluded because another publication used the same data source but provided more relevant estimates, usually with a longer follow-up or objective data: Fleming et al.,^19,20,25,40 Mundt,²⁴ and Senft and Polen.⁴¹ Two publications were excluded because they used a cluster randomized design.^18,39 All but 3 publications^46,47,49 were of sufficiently high quality to include in the meta-analysis.

Figure 1a suggests a small, insignificant positive effect of SBI on outpatient care (SMD = 0.13). The I-squared of 53.4% indicates that more than half of the variance in effect size is accounted for by between-publication differences, and the null hypothesis of homogeneity is rejected. Thus, no conclusion can be drawn about the direction or magnitude of the overall effect. Finally, there were no differences between the results of this specification and the alternative specification that included additional Project TrEAT and St. Mary's publications.

Figure 1. Meta-analytic results for outpatient and ED care.

Figure 1a. Outpatient care forest plot

Figure 1b. Emergency department care forest plot

Figure 1a and 1b Legend: *Control 1: Intervention vs. control with assessment

**Control 2: Intervention vs. control without assessment

***Drinking reasonably and moderately with self-control (DRAMs): DRAMs scheme vs. control

****Advice: Simple advice vs. control

Caption: Effect sizes are Hedges d (i.e., within-group effect sizes) with random effects. Error bars represent 95% confidence intervals. The I-squared statistic measures heterogeneity across estimates.

Figure 1b suggests a small, statistically insignificant negative effect (SMD = −0.06) for ED care. There is also low heterogeneity (I-squared = 13.9%), and the null hypothesis of homogeneity for ED utilization is not rejected, allowing for greater confidence in this result. Compared with this specification, the effect size of the alternative analysis for ED utilization increased (SMD = −0.10) and attained statistical significance. ED utilization may or may not be significantly reduced; it appears that the 2 Project TrEAT publications and additional publications from the St. Mary's study, especially Grossberg et al.,²³ are weighted heavily in the analysis and shift the confidence intervals for the ED effect size. In any case, the effect sizes do not eclipse 0.10 and must be considered marginal.

Results of the inpatient care analysis (not shown) indicated very little overall effect (SMD = 0.02, 95% CI: –0.12, 0.15). Moderate to high heterogeneity (I-squared = 69.7%, P = 0.001) prevented further interpretation of the pooled SMD.

Discussion

Systematic Review

The systematic review suggests that SBI has little to no effect on inpatient or outpatient health care utilization but may reduce ED utilization. Most publications reporting effects of SBI on health care utilization were conducted in primary care settings. Among these publications, most results were statistically insignificant for outpatient and inpatient health care utilization. Furthermore, although both statistically significant increases and decreases were reported, results were approximately evenly distributed between positive and negative effects, suggesting there is no effect. In contrast, a more consistent sign pattern was indicated for changes in ED utilization associated with SBI provided in a primary care setting. Seven of 11 publications reported decreases in health care utilization (but only 1 of the 7 was statistically significant).

Relatively few publications examined changes in health care utilization associated with SBI delivered in ED or non-ED hospital settings. The systematic review found evidence that SBI delivered in an ED setting may reduce ED utilization. All 3 publications examining ED utilization reported decreases in utilization, and 2 reported statistically significant decreases. Across all types of health care utilization, SBI delivered in non-ED hospital settings appears to have no effect on health care utilization.

Another finding of the systematic review is the inconsistent and incomplete reporting by many publications on health care utilization outcomes. For example, 25% of publications from the primary care setting reported no effect of SBI on inpatient health care utilization but provided no information on the direction, magnitude, or variance of the estimate. Although this information might seem irrelevant for small and statistically insignificant effects, it is critically important for systematic reviews and meta-analyses because it helps establish cross-publications trends that might indicate small yet meaningful effects. The absence of such information is an unfortunate casualty of space limits and reduces the ability to perform rigorous meta-analyses.

Meta-Analysis

The results of the meta-analysis support the inferences from the systematic review. For all publications, a small and statistically insignificant decrease was found for ED care. However, when multiple publications from the same underlying study (e.g., TrEAT) were included in the analysis, the ED care finding was statistically significant. Furthermore, the ED utilization analysis had minimal heterogeneity, suggesting that the average effect adequately represents the literature. Thus, although the analysis does not demonstrate a particularly robust effect, it supports a tentative conclusion of a small decrease in ED care. This result is consistent with a decrease in the likelihood of accidents and injuries resulting from reduced alcohol consumption.

No significant effect was found for outpatient or inpatient health care utilization, and the inpatient effect size was essentially zero. Although a small and potentially meaningful increase in outpatient utilization was found, the effect was insignificant, and the substantial heterogeneity across publications suggests that this effect may not adequately represent the results of the literature. The statistically insignificant increase in outpatient care and the absence of an effect for inpatient care are not necessarily unexpected. SBI was developed for risky, nondependent drinkers who are less likely than dependent users to face major chronic health care events or treatment requiring inpatient stays as an effect of their alcohol use. A small increase in outpatient utilization could signify a targeted use of treatment and support services through primary care providers or outpatient counselors, a standard message of SBI.

Heterogeneity accounts for much of the variance for the outpatient and inpatient care analyses. High levels of heterogeneity are common in SBI meta-analyses.^1,2,7,9,10 The limited number of publications prevented the use of conventional statistical tests (e.g., tests of publication bias) to examine heterogeneity further. The potential sources of heterogeneity can therefore only be discussed speculatively. Potential sources are differences in SBI setting and protocol, international regulatory differences across study settings, differences in the definition of type of care, and differences in data collection methods across publications.

The differing SBI protocols across setting and population are a potential source of heterogeneity. As noted in Ballesteros et al.,¹ 2 factors contributing to this variance are the authors' definition of risky drinking and the types of individuals included in the SBI protocol. The definition of risky drinking may or may not include heavy drinkers and may or may not have a stepped-intervention based on the level of drinking, where dosage increased with higher levels of drinking. Another distinction raised by Ballesteros et al. was whether the publications included treatment seekers and non-treatment seekers. Furthermore, there is an issue of whether the treatment effect is measured against a usual care, or control group, or against a simple advice group.¹⁰

The high number of international publications in our analysis may also contribute to the high level of heterogeneity across studies on outpatient care and inpatient care. For outpatient care, 5 publications were UK-based, 2 were Swiss-based, 1 was Australian-based, and 1 was US-based. For inpatient care, 3 publications were UK-based, 3 were US-based, 2 were Swiss-based, and 1 was Australian-based. In contrast, the ED care analysis included 4 US-based, 1 UK-based, and 1 Australian-based publication. The ED care analysis had the highest concentration of observations from 1 country and the lowest level of heterogeneity, whereas the inpatient care analysis had the least concentrated sample and highest level of heterogeneity; thus, the varying regulatory environments in the host countries may be a key source of heterogeneity across the inpatient and outpatient health care utilization results.

The definitions of the types of care (outpatient, inpatient, and ED) are another potential source of heterogeneity. There is not a standard definition across publications, so similar outcomes must be combined to find enough observable data points. For example, several publications^27,28,38 include general practitioner consultations as outpatient care. Wutkze et al.³⁴ include a more global “outpatient visits,” and Copeland et al.⁴³ use “outpatient medical stops.” ED and inpatient care had more standardized definitions across publications, suggesting that the health care utilization definition was less of a contributor to heterogeneity for those outcomes.

In addition to varying definitions, publications used varying approaches to collecting health care utilization data. Some publications used health care claims data or medical records, whereas others used self-reported measures. In the current analyses, 2 of 9 outpatient care publications, 5 of 6 ED care publications, and 4 of 9 inpatient care publications used objective health care data. The low proportion of publications using objective health care data in the outpatient and inpatient utilization analyses may contribute to the heterogeneity in those analyses.

A key limitation of this meta-analysis is the limited sample size. Of the 29 separate publications found in the systematic review, 11 – or less than 50% – were included in the meta-analysis. Several prominent and rigorous trials^18,36,39,42 were omitted from the meta-analysis; all except Bray et al.¹⁸ indicate significant decreases in utilization. Two were omitted because insufficient data were available^36,42 and two were omitted because they utilized a cluster randomized design.^18,39 Because there were not enough publications to conduct an Egger test, publication or dissemination bias was not examined.

An additional consideration is the exclusion of non-English publications. Given the high heterogeneity present in outpatient and inpatient care, including additional publications from multiple countries would further dampen any interpretation of the results. Inclusion of non-English publications for ED care could affect the results of this meta-analysis, but because the outcomes and quality of these publications cannot be readily assessed, it is difficult to surmise the magnitude and direction of those inclusion effects.

This systematic review and meta-analysis has 2 implications for the SBI field: (1) more evidence is needed on the effect of SBI on health care utilization, and (2) more evidence is needed on SBI conducted in non-primary care settings. The systematic review highlighted the lack of available data for SBI conducted in ED and non-ED hospital settings and the need for more complete and consistent reporting on health care utilization effects across all settings.

While the meta-analysis suggests that SBI may be associated with decreased health care utilization, the effect sizes are very small and insignificant. These results also support the conclusions of studies on the cost-effectiveness of SBI that most publications do not collect the necessary information for robust economic analyses, and there is not enough independent data in the field to robustly support policy. Nonetheless, results of this analysis suggest cautious optimism that SBI may reduce ED utilization. Because ED care is generally very expensive, SBI may indeed reduce overall health care costs as a result.