Questioning the Validity of the 4+/5+ Binge or Heavy Drinking Criterion in College and Clinical Populations

Abstract

Background and Aims

The terms “binge drinking” and “heavy drinking” are both typically operationalized as 4+/5+ standard drinks per occasion for women/men and are commonly used as a proxy for non-problematic (<4/<5) versus problematic (4+/5+) drinking in multiple research contexts. The Food and Drug Administration in the United States (US) recently proposed the 4+/5+ criterion as a primary efficacy endpoint in their guidance for trials examining new medications for alcohol use disorders (AUDs). Internationally, similar cut-offs have been proposed, with the European Medicines Agency having identified reductions in the number of heavy drinking days (defined as 40/60g pure alcohol in women/men) as a primary endpoint for efficacy trials with a harm reduction goal.

Analysis and Evidence

We question the validity of the 4+/5+ cutoff (and other similar cutoffs) on multiple accounts. The 4+/5+ cutoff has not been shown to have unique predictive validity or clinical utility. The cutoff has been created based on retrospective self-reports and its use demonstrates ecological bias. Given strong evidence that the relationship between alcohol consumption and problems related to drinking is at least monotonic, if not linear, there is little existing evidence to support the 4+/5+ cutoff as a valid marker of problematic alcohol use.

Conclusions

There is little empirical evidence for the 4+/5+ units per occasion threshold for “binge” or “heavy” drinking in indexing treatment efficacy. Further consideration of an appropriate threshold seems to be warranted.

The search for an ideal cut point for non-problematic versus problematic drinking has been a focus of alcohol research for the past several decades. In the college student drinking literature, the use of a cutoff for “binge drinking” (or “heavy episodic drinking”) (1) has remained prominent for decades. The initial quantification of “binge drinking” appears to have been introduced decades ago by the research team conducting the ongoing Monitoring the Future study (2), with “binge drinking” defined as having 5 or more drinks on at least one occasion in the past two weeks. This term was later adopted in 1993 by the researchers conducting the College Alcohol Study (3), leading to eventual widespread use of the term “binge drinking” to describe college student drinking patterns. In a very influential paper, Wechsler et al. (4) demonstrated that women experienced a similar likelihood of experiencing a negative alcohol-related consequence (for eight of nine consequences assessed; e.g., “miss a class”) when they reported drinking 4 US standard drinks (with one standard drink equal to 14g of alcohol) per occasion on average as men who reported drinking 5 US standard drinks per occasion on average. Accordingly, binge drinking became defined as consuming 4+/5+ standard drinks on a single occasion for women/men.

More recently, the National Institute on Alcohol Abuse and Alcoholism (NIAAA) in the US adopted two definitions of binge/heavy drinking: a) consuming 4+/5+ drinks for women/men within a two hour period, or b) drinking to a level that brings one’s blood alcohol concentration (BAC) to .08 g/dL. NIAAA claims that a BAC of .08 g/dL “typically occurs after 4 drinks for women and 5 drinks for men---in about 2 hours” (NIAAA, 2004), adding the time qualifier to reflect elevated BACs that may be more indicative of problematic use than quantity cutoffs alone. However, based on multiple BAC calculators (5,6) and an actual field test (7), there does not appear to be any serendipitous association between a BAC level of .08 and the 4+/5+ cutoff (calculations available from first author), and the 4+/5+ criterion without the 2-hour time qualifier was more strongly correlated with alcohol-related consequences than the 4+/5+ criterion with the time qualifier in a college student sample (8). The 4+/5+ cutoff for “heavy drinking” has also received traction in being applied with clinical populations, with no heavy drinking days identified as a primary endpoint for alcohol clinical trials by the Food and Drug Administration (FDA) in the US (9,10) and reductions in heavy drinking days (defined as 4+/6+ 10g ethanol drinks) as a primary endpoint for alcohol clinical trials by the European Medicines Agency (EMA) in Europe (11).

Although various cutoffs have been proposed and/or examined to define binge or heavy drinking in the US (1,12–16) and internationally (11,17) (see Table 1 for a brief summary), the 4+/5+ criterion is pervasive. Although most of our critiques apply to any of these cutoffs, we focus on the 4+/5+ criterion specifically. We discuss issues with the use of binge/heavy drinking as a primary outcome in studies examining college student alcohol use and clinical populations with alcohol use disorders (AUD). Several critiques have been levied against both the use of the “binge drinking” term (18,19) as well as the 4+/5+ definition (20), including a special issue published nearly 15 years ago (21). To expand on these prior critiques, we focus our review on issues related to the 4+/5+ definition that have received less attention in the field: limited predictive validity and clinical utility, ecological bias, and a false dichotomization.

Table 1.

Cutoffs for binge/heavy drinking definitions for women/men

Cutoff	Drinking Span	Occurrence Span	1st Studies/Sources
5+	1 occasion	Past 2 weeks	MTF, CAS
4+/5+	1 occasion	Past 2 weeks	Wechsler et al., 1995
4+/5+	2 hour period		NIAAA, 2004
.08+ g/dL BAC	1 occasion		NIAAA, 2004
4+/5+	1 day	Follow-up period	FDA, 2006; Falk et al., 2010
4+/5.7+	1 day		Jefferis et al., 2005
6+/7+	1 day	Past 90 days	Read et al., 2008
8+/10+	1 occasion	Past 2 weeks	White et al., 2006
10+	1 day	56 days	Jackson, 2008

Note. Read et al. (2008) referred to this as ‘heavy’ binge drinking; White et al. (2006) examined double and triple the binge threshold; Jackson (2008) tested various thresholds and did not advocate for a specific threshold

Ongoing Use of the 4+/5+ Definition

Perhaps the best argument for continuing to assess the 4+/5+ pattern of drinking is that it has been assessed “for several decades” (p. 317), a point admitted even by critics of this binge drinking definition (19). Yet, the continued application of the 4+/5+ definition beyond its initial use in epidemiological and college student samples is problematic. Wechsler and Nelson (22) defined binge drinking conceptually “as the consumption of a sufficiently large amount of alcohol to place the drinker at increased risk of experiencing alcohol-related problems and to place others at increased risk of experiencing secondhand effects” (p. 287). They also noted that “in the context of full-blown alcoholic drinking…five drinks seem comparatively small” (p. 287). However, the Food and Drug Administration (10) has now accepted the 4+/5+ definition for a “heavy drinking day” as a primary endpoint for alcohol clinical trials, with treatment success defined by the percent of subjects with no heavy drinking days (PSNHDDs) in a follow-up period (9). The adoption of a 4+/5+ definition of heavy drinking for a clinical trial is a step in the direction of allowing some drinking after treatment, which is considered an improvement from a focus on abstinence-only as the primary endpoint. However, it is noteworthy that a single drinking episode of 4+/5+ drinks as an indication of alcohol treatment failure was never intended by Wechsler and colleagues (22) or other studies that have sought to validate the 4+/5+ definition in college student samples. Further, Gmel and colleagues (23) highlighted that a single heavy drinking episode yields little predictive utility, according to their review of international research on the association between heavy drinking with fetal outcomes, coronary heart disease, and injury.

Limited Predictive Validity and Clinical Utility

Several articles by Wechsler and colleagues have argued for the validity of the 4+/5+ criterion based on the fact that individuals who engage in such “binge drinking” experience more alcohol-related problems than those who do not (3,4,24). However, this is not a true validity test. Nearly any cutoff that distinguishes between heavier versus lighter drinkers will find individuals above the cutoff to experience more problems than the individuals below the cutoff. For the binge drinking criterion to warrant consideration, we should be able to demonstrate that it offers unique predictive validity, or incremental validity, when controlling for other measures of alcohol consumption (e.g., continuous frequency and quantity measures). For example, although some research has suggested that ≥3 drinks is a helpful cutoff for screening individuals for an AUD (25), there is little evidence that such cutoffs predict future experience of alcohol-related problems. Across two samples of college students, Borsari et al. (26) not only found that weekly quantity measures during either a typical or peak drinking week were more strongly correlated with alcohol-related problems than binge drinking frequency, but also that binge drinking frequency did not significantly predict alcohol-related problems when controlling for either typical or peak weekly drinking and blood alcohol level. Additionally, Read et al. (27) examined the association between binge drinking status and other alcohol-related outcomes (e.g., drinks per drinking day, typical estimated blood alcohol content, alcohol-related consequences) and found that only “heavy binge” drinkers (6+/7+ for women/men) experienced significantly more alcohol-related consequences than non-binge drinkers. In other words, not only was the 4+/5+ cutoff determined without the strongest methodology or use of incremental validity, but also other measures of drinking behavior have yielded greater predictive validity than the 4+/5+ cutoff. Accordingly, the predictive validity of the 4+/5+ cutoff is limited and, as such, is not necessarily appropriate to use even in college student populations as originally proposed.

In addition to limited predictive validity, there is no clear basis for the 4+/5+ cutoff related to clinical utility. Specifically, it has not been demonstrated that this criterion has greater sensitivity and/or specificity than any alternative measures in the prediction of important secondary outcomes (e.g., alcohol consequences, quality of life). In a clinical sample, Witkiewitz (28) found that temptation to drink during treatment had higher accuracy and sensitivity in the prediction of alcohol-related outcomes at 1- and 3-year follow-ups than did heavy drinking (using the 4+/5+ cutoff). Falk et al. (9) demonstrated that the percent subjects with no heavy drinking days (PSNHDDs) outcome detected differences between medication and placebo groups, and found that these effect sizes were comparable to effect sizes for other alcohol-related outcomes (e.g., percent days abstinent, PDA). Yet only a few studies have attempted to validate the operationalization of “heavy” drinking at 4+/5+, specifically for alcohol treatment seeking individuals with AUD. In a sample of medical patients, Kranzler and colleagues (29) concluded that frequently drinking six or more drinks per occasion was associated with alcohol-related problems. Similarly, in a prevention trial for heavy alcohol use, Sanchez-Craig and colleagues (30) found that those who had alcohol-related problems drank on average 6.3 drinks per drinking day (males) and 4.2 drinks per drinking day (females).

Defining binge drinking using a one-size-fits all cutoff based on early research showing college students were at increased risk of experiencing largely college student-specific drinking consequences may not only inappropriately transplant college student definitions to clinical populations, but also essentially treats all alcohol related consequences as equal. As recently demonstrated by Lane and Sher (31), treating all alcohol-related problems as equal overlooks clinically meaningful differences between each alcohol-related problem (e.g., the difference between experiencing tolerance versus experiencing failures to fulfill role obligations due to drinking). Accordingly, using the 4+/5+ definition may yield little clinical insight into one’s problem severity or risk for harm.

In addition to yielding little clinical insight, arbitrarily defining binge/heavy drinking as 4+/5+ drinks for women/men is insensitive to individual differences in body mass and cultural context, as noted by other critics of quantitative cutoffs (32,33). For example, a 250 lbs. male who drinks six standard drinks over two hours (approximate BAC = 0.058) will be much less impaired and likely experience fewer problems than a 175 lbs. male who drinks the same amount over the same amount of time (approximate BAC = 0.097). The existing definition also assumes all individuals who drink 4+/5+ standard drinks per occasion are at increased risk of harm without any consideration of culture or other contextual factors. As noted by Gual and colleagues (1999), the quantity of alcohol consumption may vary based on country of origin (e.g., wine-drinking countries such as Spain), so considerations of culture or other contextual factors may be highly important when determining if someone is at increased risk of alcohol-related harms based solely on the quantity of alcohol consumed.

Retrospective Self-Reports and Ecological Bias

Although large epidemiological studies have made several contributions to the field, one contribution that they cannot make is to provide suggestions for how many drinks an individual should have on a single drinking occasion. First, these extant studies have used retrospective self-reports of drinking, which are known to be associated with significant recall biases in a time frame as short as one week (34,35). Second, they have targeted assessment of both alcohol use and alcohol-related problems over relatively large periods of time. A serious limitation of this approach is that it leaves researchers unable to determine when these negative consequences are occurring in reference to alcohol use. For example, although Wechsler and colleagues have largely focused on average level of drinking or the number of binge drinking episodes (4,36), several investigators have demonstrated the incremental validity of peak or heaviest drinking measures in the prediction of negative consequences (37,38). Researchers have assumed that more negative consequences are occurring on peak drinking days relative to ‘typical’ drinking days, but this assumption cannot be tested with retrospective data. Third, providing individual, event-level recommendations using between-subject data is an example of the ecological fallacy, or ecological bias (39). Ecological bias occurs when data from one level of analysis (e.g., between-subjects) are used to make inferences regarding another level of analysis (e.g., within-subjects). Within-subject comparisons are more useful for making clinical and public health recommendations than between-subject comparisons.

All of the problems reviewed above can be addressed through the use of data from daily diary or ecological momentary assessment (40) methods. All alcohol consumption occurs at the event-level; thus, to gain a better understanding of how amount of alcohol use (among several other factors) relates to alcohol-related consequences, event-level data are absolutely necessary. For example, if a man consumes 5 or more drinks on a Friday night, is he at increased risk for consequences as compared to when he consumes 4 drinks or less on a Friday night? This question is most appropriately answered with event-level data. Only event-level data can correctly link specific amounts of consumption on a particular day to alcohol-related problems experienced in close proximity to the drinking event (e.g., next day hangover). Event-level recommendations can only follow from event-level data, so meaningful cutoffs would almost certainly have to follow from such data. The current applications of the 4+/5+ cutoff have relied upon an assumption that the drinking occasions involving 4+/5+ drinks directly place one at increased risk of problems, but such an assumption is based on potential ecological bias and future research is needed to empirically evaluate this assumption using event-level data.

Monotonicity Suggests False Dichotomization

A final critique of the 4+/5+ definition can be levied toward any cutoff based on alcohol consumption, and this critique has been raised by several researchers (18,19,41). There is strong evidence that the relationship between alcohol use and negative consequences from drinking is at least monotonic if not linear in both college student (4) and clinical populations (28). In other words, in the absence of examining any other factors, any increase in alcohol consumption is associated with an increase in the likelihood of experiencing alcohol-related problems, or negative consequences from drinking. To the extent that the association between alcohol use and negative consequences from alcohol approaches a non-linear step function (see Figure 1), a meaningful cutoff can be created at the inflection point where risk increases substantially. However, insofar as the association between alcohol use and negative consequences approaches a linear association (see Figure 1), any cutoff will be arbitrary and in our literature review we have found no evidence of such an inflection point. For example, Wechsler et al. (4), which was used to create the 4+/5+ cutoff, shows linear associations between level of drinking and alcohol consequences for both men and women. Thus, the data used to create the 4+/5+ definition are linearly related and would not provide strong support for the validity of any cutoff. It is unsurprising that more recent research has failed to identify any meaningful cutoff (1,42).

Figure 1.

Demonstration of two hypothetical associations between alcohol use and negative consequences from drinking. As illustrated, a stepwise relationship would clearly distinguish endorsement of alcohol-related problems (hypothetical y-axis) for those who drink < 4 drinks from those who drink 4+ drinks whereas a linear relationship (which is generally found in research) between alcohol-related problems and quantity of drinks fails to provide a meaningful cutoff.

Using a cutoff with linearly related data is essentially falsely dichotomizing a continuous measure. Dichotomization of continuous measures makes a tenuous assumption that those in one group (i.e., non-binge drinkers) are qualitatively different from those in the other group (i.e., binge drinkers). Continuous variables in statistical models rarely perform worse than dichotomous variables, but dichotomous variables usually perform worse than continuous variables (43). Dichotomization of a continuous measure can result in a loss of statistical power, making statistical tests less sensitive to detecting existing effects, which has serious implications when these measures are used to quantify the effects of preventions/interventions. In other words, false dichotomization increases the likelihood of a Type II error, which may be problematic for detecting intervention effects.

Moreover, relying upon false dichotomies may result in inaccurate labeling of individuals as “problem” drinkers, leading at best to failing to identify problematic drinkers that fall under the cutoff (i.e., low sensitivity) and, at worst, to misidentifying non-problem drinkers as problem drinkers (i.e., low specificity). Such misidentification may damage the relationship between client and provider and/or tarnish the public’s perception of researchers and clinicians. This byproduct is especially salient when the 4+/5+ cutoff is applied to clinical samples to evaluate treatment success, where “moderate drinking” has been defined as 3 or less drinks per drinking day for women and 4 or less drinks per drinking day for men in clinical trials (e.g., (44)). Accordingly, a single drink distinguishes an individual’s drinking occasion as a moderate versus a binge drinking episode in clinical populations, which means one’s status as a treatment “success” or “failure” may hinge upon one standard drink.

Summary of Critiques

The aforementioned critiques provide arguments that the current definition of binge drinking as 4+/5+ drinks for women/men may be an overreliance upon a cutoff that was derived from college students and their increased risk of experiencing alcohol-related consequences of most relevance to college students. As this definition has been carried forth with attempts to improve the cutoff by considering estimated blood alcohol concentration levels of .08, the 4+/5+ cutoff has been further reified. Further, the cutoff has been applied to clinical populations to identify treatment “failure” with little empirical evidence that 4+/5+ drinks per drinking occasion is indicative or predictive of problem severity or clinical outcomes in AUD samples.

In addition to the historic problems that have emerged from defining binge drinking with the 4+/5+ rule, there are a number of other shortcomings of this definition. Specific limitations we have identified include that the 4+/5+ definition has limited predictive validity across samples by not examining incremental validity; lacks clinical utility in that the definition is sometimes applied arbitrarily and fails to consider culture and other contextual factors; assumes alcohol-related consequences only occur directly from higher-drinking episodes, despite a lack of event-level data; and relies on a single cutoff from continuous data, which may unnecessarily lose important nuances to data, inflate Type II error, and create a false dichotomy from continuous data.

Potential Alternatives

Future research should move away from a single cutoff, especially when identifying individuals as treatment “failures” or “successes.” One obvious alternative to using a binary outcome is to use the original continuous or count variable that is typically dichotomized to create the binary outcome. Using a variable in its original metric provides the most statistical power, although is criticized for not having clinical significance. Statistical tests are apt to determine whether a change is statistically significant (i.e., a change greater than expected by chance or due to error), and effect size measures can be used to determine if the magnitude of these changes are practically or clinically relevant. Given the monotonic association between alcohol consumption and negative consequences from drinking, one may expect that any statistically significant reduction in alcohol consumption would have a downstream effect on reducing alcohol consequences.

Another alternative to using a consumption-based cut-off that is presumed to be related to alcohol problems is to simply use alcohol problems as the focal outcome. It is not unreasonable to suggest that the ultimate outcome to evaluate the reduction of harm from drinking is the negative consequences from drinking. We may define negative consequences broadly including psychological, physical, and/or social outcomes, and the focus of a particular study will depend on the goal of the intervention. In one population, the reduction of self-reported alcohol-related problems may be the focal outcome, whereas in another study it may be liver enzymes. Our main point is that rather than relying on a consumption-based cut-off as a proxy for risk, we ought to put more emphasis on measuring and evaluating change in risk directly.

If consumption-based cut-offs were absolutely necessary (e.g., other outcomes could not be assessed), we argue that the use of EMA designs or real-time objective alcohol monitoring (e.g., transdermal alcohol monitoring, Leffingwell et al., 2013) provides the only path to creating valid cut-offs for individuals. Not only do these designs allow the examination of how specific alcohol levels relate to alcohol consequences in near real-time, but they also have the potential to be used to provide more individualized cut-offs. For example, using machine learning, a mobile application collecting EMA data could “learn” how a complex set of interacting factors result in problematic alcohol consumption for a particular individual, which could include level of consumption, speed of consumption, drinking context, and many other factors. Consistent with the goals of personalized medicine (Hamburg & Collins, 2010), such an approach could be used to classify drinking events as “safe” or “risky” differently for each individual, and provide interventions uniquely for each individual based on the specific factors that determine whether their drinking is “safe” or “risky.”

Conclusion

Although some of the potential alternatives to consumption-based cutoffs need to be further studied prior to implementation, it is important that researchers critically evaluate any cutoff they create and work toward developing a cutoff that is specific to the population and desired outcomes. Specifically, researchers should consider cutoffs that relate to alcohol-related consequences and that are meaningful for the population being evaluated. For example, if clinical populations are the focus, then researchers should consider levels of drinking that are associated with clinically significant improvement in clinical outcomes associated with alcohol consumption (e.g., quality of life, functioning).