Conducting economic evaluations of screening and brief intervention for hazardous drinking: Methods and evidence to date for informing policy

Abstract

Issues

Many policy review articles have concluded that alcohol screening and brief intervention (SBI) is both cost-effective and cost-beneficial. Yet a recent cost-effectiveness review for the United Kingdom’s National Institute for Health and Clinical Excellence suggests that these conclusions may be premature.

Approach

This article offers a brief synopsis of the various types of economic analyses that may be applied to SBI, including cost analysis, cost-effectiveness analysis, cost-utility analysis, cost-benefit analysis, and other types of economic evaluation. A brief overview of methodological issues is provided, and examples from the SBI evaluation literature are provided.

Key Findings, Implications, and Conclusions

The current evidence base is insufficient to draw firm conclusions about the cost, cost-effectiveness, or cost-benefit of SBI and about the impact of SBI on health care utilisation.

Introduction

Numerous studies have shown that screening and brief intervention (SBI) is effective in reducing alcohol use and associated problems for most populations at risk [1–9]. Furthermore, several review articles have concluded that alcohol SBI is a cost-effective preventive service [8, 10–11], and many policy makers are advocating that SBI be used as part of standard clinical practice (e.g., [12–14]). Despite the widespread perception that SBI is cost-effective, the scientific evidence base supporting this conclusion is relatively weak, and more research is needed before definitive policy recommendations can be made [15]. This article presents an overview of the types of economic analysis methods that are germane to SBI research.

Alcohol screening is typically based on a structured instrument (e.g., the Alcohol Use Disorders Identification Test [16]). The brief intervention (BI) is any therapeutic or preventive consultation of short duration (one to five sessions) undertaken by a health care professional [17–18]. It typically comprises assessment, feedback, information, advice, and the provision of self-help materials [19]. The majority of the SBI literature focuses on one key target population, hazardous drinkers [20], typically defined as people who drink above guidelines for safe drinking but who would not be considered dependent. The dimensions by which SBI may vary include the screening tools used, the delivery approach, the setting, the country or region, and the staff delivering the intervention. These dimensions will determine both the cost of the intervention and its impact on outcomes.

In an era of limited resources in most health care systems, decision makers frequently demand that interventions and treatments be justified fiscally as well as clinically. Although a single session of SBI for hazardous alcohol use may be expected to be relatively cheap, SBI as a public health program may have a significant fiscal impact when applied to a large population. Furthermore, models of SBI implementation will vary considerably in their implications for resource use, based on the duration and number of BI sessions and on the qualifications of the interventionist. Economic evaluation of SBI is therefore needed because it involves assessing resource allocation implications of competing health care programs. The analytic approaches in economic evaluation determine the cost of SBI and/or the value of outcomes in money terms or some other standard unit. Economic evaluation helps decision makers set budgets and chose among the myriad alternative uses for scarce health care resources. Key studies and texts for conducting economic evaluation on SBI are noted throughout this review. Reference texts for conducting economic evaluation in health care more generally are Gold et al. [21] and Drummond et al. [22]. Thorough reviews of specific types of economic evaluation when applied to SBI can be found in Babor et al. [23] and Latimer et al. [15].

Economic Evaluation: Methods and Summary of Evidence

There are five broad approaches to economic evaluation (Table 1). The first four approaches require estimating the value of SBI and are inter-related (Figure 1): cost analysis, cost-effectiveness analysis (CEA), cost-utility analysis (CUA), and cost-benefit analysis (CBA). Cost analysis estimates the value of resources used to deliver SBI. CEA compares the relative costs and effectiveness of two or more alternative programmes using the incremental cost-effectiveness ratio (ICER), which expresses the additional cost required to improve effectiveness by one unit. CUA is a form of CEA that has a standardised metric of effectiveness that indirectly reflects utility (or satisfaction). CBA estimates the value of all relevant outcomes of SBI and weighs that value against its costs. The fifth approach to economic evaluation examines the impact of SBI on one or more economic outcomes (e.g., number of outpatient visits).

Table 1.

Key considerations, advantages, and disadvantages of approaches to economic evaluation

Approach	Key Considerations	Advantages	Disadvantages
Cost analysis	Preferable to use systematic data collection tools, such as the Substance Abuse Services Cost Analysis Program (SASCAP)	Allows policy makers to identify cost drivers and draft budgets.	Completing cost tools can be burdensome for researchers and programme staff.
Cost-effectiveness analysis (CEA)	Usefulness depends on which outcome is analyzed. Cost-effectiveness ratios for different treatment outcomes may have different policy implications.	CEAs can yield a ranking of competing alternatives.	The approach does not provide information on the value of a programme independent of alternatives.
Cost-utility analysis (CUA)	Applies to CEA a common metric (e.g., the quality-adjusted life year [QALY]).	QALYs can compare a wide range of diverse programmes.	QALYs are difficult to measure, especially for services expected to have small effects.
Cost-benefit analysis (CBA)	CBAs measure both costs and outcomes in monetary terms, which helps comparisons of screening and brief intervention (SBI) with other types of investments in health and well-being. In practice, CBAs may only put a value on some but not all outcomes.	CBAs can allow direct comparison of costs to benefits. May give clearer guidance than CEAs on which treatment to adopt.	Many outcome measures cannot be expressed in monetary. Benefits of some interventions might be measured more readily than benefits of others, and some benefits may manifest only over several years after treatment.
Analysis of impact on economic outcomes	Typically focuses on the degree to which health care utilization is changed by SBI.	Does not require monetizing either SBI or the outcome.	Unless SBI and outcomes are expressed in money terms, the implications for budget and planning may not be clear.

Figure 1.

Connections between cost, cost-effectiveness, cost-utility, and cost-benefit analysis

Within each approach, the perspective determines which stakeholders’ costs, outcomes, and benefits are included. A societal perspective—which includes all stakeholders’ costs and outcomes—is often recommended and used (e.g., [21]). However, narrower perspectives, such as the health care system, are often useful for forming policy and determining whether to adopt services [22].

Cost analysis

A cost analysis is the first step in a full economic evaluation of SBI and provides critical information beyond its contribution to CEA, CUA, or CBA [22]. A detailed cost analysis of SBI can be used to accurately project budgets under different configurations of resources. Cost analysis by itself may not incorporate information on the impact on outcomes of SBI. Thus, cost minimisation analysis—choosing an intervention based on lowest costs alone—is not recommended as a valid method for making policy recommendations [22].

Evaluators may choose to add future health care costs to the costs of the intervention. This approach is often taken in the broader health economics literature [24] and sometimes taken in the literature on substance use interventions [25]. Alternatively, future health care costs may be omitted from the cost of the intervention and instead counted as a benefit in a CBA as an averted health care cost. This approach is taken by many studies on substance use and several studies on SBI (e.g., [16, 26–31]). A recent study uses both alternatives, by excluding future health care costs from the intervention cost in the CEA, but including them in the intervention cost of a CUA [32]. Because future health care costs may be legitimately used differently in economic evaluations, it is important that researchers report how future health care costs are included (see Drummond et al. [22], pages 17–22).

Two features of SBI are particularly important for conducting cost analysis. First, data on the cost of SBI may require primary data collection methods. In some countries, such as the United States, reliable data on the provision of SBI may not be available in administrative records because it is not a reimbursable service in every state or health plan. Second, regardless of how cost data on SBI are collected, it is important that they be as accurate as possible. SBI is brief by design. Thus, estimating the duration of screening to be 2 minutes rather than 1 minute, for example, doubles the cost estimate of screening. This will clearly affect the budget for policy makers trying to fund a universal screening programme. Systematic methods to estimate all resources used in SBI help ensure the accuracy and comparability of cost estimates. An example among several available protocols and versions [33–37] is the Substance Abuse Services Cost Analysis Program (SASCAP) [38].

Estimates of the cost of the intervention component alone of SBI (i.e. excluding future health care costs) [25–26, 29, 39–45] vary so widely as to prohibit drawing conclusions about the typical cost of providing SBI. The variation in cost is illustrated by two studies [29, 41] in the setting with the most studies, the primary care/general practitioner setting. The cost of screening varies from $0.52 [41] to $181.41 [29], and the cost of BI varies from between $3.24 and $4.29 (depending on the specific intervention) [41] to $89.66 [29].

The literature provides limited guidance on the source of variability in cost estimates, in large part because there are so few stand-alone cost studies in the peer-review literature. Most published cost estimates are presented as part of a CEA, CUA, or CBA and thus provide insufficient detail on the cost analysis methods. Thus, it is currently not possible to determine whether differences in costs are driven by differences in study methods, settings, staffing, screening tool, delivery, or target population.

Cost-effectiveness analysis

CEA compares two or more treatments using the ICER—the difference in costs between two interventions divided by the difference in outcomes [21]. The ICER reveals how much more it costs to achieve one additional unit of outcome with one alternative compared to another. For example, an ICER could show that SBI Programme A costs $1,000 per person more than Programme B to reduce average drinks per week by one. The ICER is often compared to willingness to pay, which is the stakeholder’s implicit value of the outcome. The most desirable intervention among a set of mutually exclusive choices is the most effective intervention with an ICER that is less than the willingness to pay for the intervention. For a clear and concise summary of the decision rules to use when interpreting CEAs, see Bala and Zarkin [46].

Two key issues when conducting CEA are estimating sampling variation in the estimates and choosing the outcome to be analysed. Because the ICER is the ratio of two random variables, incorporating statistical variation in costs and effectiveness into CEA to obtain a confidence interval or standard error is not straightforward. Approaches include using bootstrap techniques to calculate a confidence interval or standard error [47] or using cost-effectiveness acceptability curves (CEACs) (e.g., [48–50]) when comparing two interventions and cost-effectiveness acceptability frontiers (CEAFs) when comparing three interventions or more (e.g., [51]). CEACs and CEAFs graphically represent the probability that a particular intervention is optimal over a range of willingness-to-pay values.

With regard to choosing the outcome to be analysed, any relevant outcome, such as drinks per week, can be used if it is relevant for all interventions under consideration [22]. However, because SBI may yield several worthwhile outcomes, the ICER for one outcome measure may yield a different policy implication than the ICER for another outcome measure [52]. The literature on substance use has yet to agree on a standard, aggregate outcome and has yet to adopt utility measures in CUA (discussed below) [53].

Although several review articles conclude that SBI is cost-effective [8, 10–11], CEA results vary considerably. This variation reflects the issues underlying the supporting cost analyses, previously discussed, and the fact that many studies use different metrics for outcomes. The most commonly used outcome in CEA studies of SBI is the average number of drinks per week [16]. Kunz et al. [43] and Barrett et al. [25] both use average drinks per week as a metric and are set in an emergency department; however, the two are set in different countries. Kunz et al. is set in the United States and finds an ICER of $273 per 1-drink reduction, whereas Barrett et al. is set in the United Kingdom and finds an ICER of $39.

Cost-utility analysis

CUA is a form of CEA that expresses outcomes in a common metric, typically the Quality Adjusted Life Year (QALY) [21]. QALYs assign a quality weight to each additional year of life generated by a treatment, where 1.0 indicates perfect health and 0 indicates death. Although QALYs can be used to compare a wide range of programmes with each other, they can be difficult to estimate for services that are expected to have only small effects on the patient’s quality of life in the near term or length of life over the long term. QALY weights have yet to be constructed for every substance use measure, including hazardous drinking. Moreover, it is unlikely that any utility measure would be able to capture the impact of alcohol use on important societal outcomes, such as crime [52].

CUA often relies on modeling to obtain QALY impacts. The outcome available in studies is typically alcohol consumption 1 year after the intervention, and this must then be converted into a long-term impact [54]. A commonly used method is Markov modeling, which models transitions between a limited number of states (e.g., [9]). More sophisticated modeling allows for individual-level heterogeneity [55]; such modeling has recently been used in the economics of substance use [56], but not specifically for SBI. It should be noted that the conclusions drawn from modeling will depend on the validity of the model.

A recent review of CUA of SBI for the United Kingdom’s National Institute for Health and Clinical Excellence (NICE) [15] bases recommendations on four studies that met the review criteria [9, 11, 57–58]. The report concludes that, based on standard willingness-to-pay values, SBI in the United Kingdom likely would be cost-effective in primary care settings but that sufficient evidence does not exist to make a similar conclusion with regard to emergency department and hospital settings.

Cost-benefit analysis

CBA directly compares the costs of SBI to the value of all relevant outcomes (e.g., the value of avoided health care services or car crashes) and thus can answer the question of whether an intervention is worth the cost [22]. CBA may also be conducted by multiplying the estimated QALYs gained from an intervention by willingness-to-pay estimates (discussed in the CEA section above) and comparing the result to the intervention’s costs.

Conceptually, because CBA allows direct comparison of costs to benefits, it may provide clear guidance on which interventions should be adopted independent of the alternatives. Practically, however, CBAs are often difficult to implement. It is challenging to estimate values for all relevant outcomes of the interventions because many clinically relevant outcomes, such as reduced alcohol use, may not be easily expressed in monetary terms. Furthermore, it may not be possible to consistently measure all relevant outcomes of an intervention.

CBA results are typically computed as either net benefit estimates or benefit-cost ratios. To calculate the net benefit, the sum of costs is subtracted from the sum of benefits; to calculate benefit-cost ratios, benefits are expressed as a percentage of costs [22]. Perhaps because of the practical challenges of CBA, only four publications from two separate projects conduct a formal CBA. Although all four suggest that SBI is cost-beneficial [26, 28–29, 59], the limited number of studies suggests caution should be used in drawing policy conclusions.

Analysis of the impact of SBI on economic outcomes

Much of the current literature on the economic evaluation of SBI examines the impact of SBI on economic outcomes, primarily health care utilisation [42, 60–72]. All but one of these studies keep utilisation in its natural units, such as outpatient visits, rather than convert utilisation into costs. There is no comprehensive review of the impact of SBI on health care utilisation. Two reviews focusing on the impact of SBI on alcohol use secondarily address health care utilisation. The first examines four studies in emergency department settings and suggests that SBI reduces health care utilisation [64]. The second examines 30 studies in primary care settings and found only 1 study where SBI was associated with reductions in health care utilisation [73].

A related approach to economic evaluation, cost-offset analysis, limits the perspective of analysis to the health care system. The analysis assesses the cost of the intervention against the value of any reductions in health care utilisation. Although cost-offset analysis is well established in the behavioral health arena broadly (e.g., [74–77]), few studies assess the cost-offset of SBI specifically. The cost-offset findings for SBI are mixed. For example, one study set in the emergency room finds health care cost savings [42], whereas another that focuses on older adults in primary care settings finds little effect [29]. Other studies wrap possible medical provider health care cost savings into a broader perspective CEA, CUA, or CBA (e.g., [11, 25, 29]), discussed previously. It is currently unclear the degree to which health care cost savings drive the results of these broader analyses.

Discussion

Economic evaluation methods are increasingly being used to support policy efforts to disseminate alcohol SBI. This paper has presented a brief overview of the five types of economic evaluations and the SBI literature for each. Despite a widespread perception among policy makers that SBI is cost-effective and cost-beneficial, the peer-reviewed literature supporting this conclusion is largely insufficient [15]. Three limitations of the evidence base for making policy decisions are most notable. First, the cost analyses that underpin the CEA and CBA analyses provide a very large range of cost estimates that makes it difficult for decision makers to appropriately budget services. These studies typically do not provide sufficient details about their cost analysis methodology to allow an informed understanding of why SBI cost estimates vary so widely. The increasing availability of online appendices that are subject to peer-review scrutiny is perhaps one solution to providing the necessary detail on intervention costs.

A second limitation of the current literature is that CEA studies of SBI do not use consistent and uniform outcomes, such as QALYs. Caution should always be applied in basing policy solely on potential cost savings to the health care system. Balancing cost implications against a consistent health outcome measure using CEA or CUA is thus needed. The variation in outcomes coupled with the variation in the underlying cost estimates, however, prohibits drawing firm conclusions about the cost-effectiveness of SBI in all health care settings or making comparative assessments of one SBI model to another. To help guide policy, it is recommended that a core set of outcomes be determined and used.

A third limitation is that conclusions of policy review articles are typically based on a handful of studies that show SBI to be cost-beneficial while ignoring other studies, many of which fail to find statistically significant cost savings associated with SBI. Although many economic evaluations focus on the impact of SBI on health care utilisation, there has been no comprehensive, systematic review or meta-analysis of these studies. Health policy requires such a study so that resources can be allocated appropriately.