Skip to main content
NCBI home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2019 Sep 1.
Published in final edited form as: Drug Alcohol Rev. 2018 Jul 9;37(6):774–781. doi: 10.1111/dar.12839

Effects of a Responsible Retailing Mystery Shop Intervention on Age Verification by Servers and Clerks in Alcohol Outlets: A Cluster Randomized Cross-Over Trial

Joel W Grube 1, William DeJong 2, Maureen DeJong 3, Sharon Lipperman-Kreda 4, Brad S Krevor 5
PMCID: PMC6120763  NIHMSID: NIHMS975454  PMID: 29984430

Abstract

Introduction and Aims

Clerks and servers at alcohol establishments often fail to verify age. Using mystery shoppers to provide performance feedback is one approach to improving age verification. Few controlled studies have evaluated mystery shop interventions. This paper reports on a cluster randomized cross-over trial of a mystery shop intervention in 16 communities in four U.S. states.

Design and Methods

The intervention comprised monthly mystery shops after which clerks and servers received immediate feedback to reinforce age verification. Managers received monthly reports. Communities in each state were matched into pairs and, following a 3-month baseline, were randomly assigned within each pair to receive the intervention beginning after the fourth (Early Intervention) or the tenth (Delayed Intervention) mystery shop. On average, 17 (range = 14–20) randomly selected on-premises and 18 (range = 11–23) off-premises outlets participated in each community (N = 557).

Results

Fixed effects multi-level logistic regressions indicated that the intervention led to a two-fold increase in the odds of age verification, OR = 2.05; p < .001; 95% CI = 1.76, 2.39. Overall, ID-checking increased from an average of 80% pre-intervention to 94%–96% by the end of the program. Significant effects were found for on-premises, OR = 1.79; p < .001; 95% CI =1.43, 2.24, and off-premises establishments, OR = 2.29; p < .001; 95% CI = 1.86, 2.82.

Discussion and Conclusions

Mystery Shop interventions can increase age verification for alcohol purchases and may be an effective supplement to compliance checks and responsible beverage service programs.

Keywords: Alcohol, Underage, Sales, Mystery Shops, Prevention

Introduction

Most countries, states, or provinces set a minimum legal age for purchasing alcoholic beverages [1]. Higher purchase ages are associated with lower consumption and fewer alcohol-related problems among underage youth [2, 3]. Nonetheless, a majority of underage youth report easy access to alcohol [4, 5]. Notably, research consistently shows that significant numbers of clerks and servers in retail establishments fail to properly verify age and sell alcohol to minors [6–15], underscoring the need for interventions that can reduce youth access to alcohol through commercial sources [16–18]. This paper presents the results of a randomized trial evaluating a mystery shop intervention designed to increase ID-checking for the sale and service of alcoholic beverages.

There are a number of strategies for preventing underage sales of alcoholic beverages. Law enforcement compliance checks are one effective approach [19–22], but they are expensive to implement, labour intensive, and the effects decay relatively quickly when enforcement activities cease [19, 22]. In the absence of dedicated funding and public support, compliance checks are seldom sustained or conducted frequently enough to have more than short-lived effects [23–25].

Responsible beverage sales and service programs (RBS) focus on the development and implementation of outlet policies to prevent sales to minors and training staff in responsible sales and service practices. Research indicates that RBS is received favourably by retailers, associated with retailer-reported changes in policy [26], and may reduce the likelihood of alcohol sales to minors [27], although the evidence is mixed [20, 22]. Increases in age verification resulting from RBS programs, however, may dissipate relatively quickly in the absence of ongoing training and incentives [22]. Although use of electronic age-verification systems can increase ID-checking and compliance [28], research on their effectiveness is mixed, possibly because of inconsistent use by staff [29]. Remote ID-checking technology where a validator judges the customer’s age through a video camera link is promising [30] but has not been implemented widely.

A key element missing from many programs is regular feedback to staff and managers about ID-checking behaviours. An approach that fills this gap is to use mystery shoppers who attempt to purchase alcohol and then provide feedback on whether the clerk or server asks for ID. Because of legal considerations, alcohol mystery shops in most states in the U.S. must be conducted with young legal-age customers (e.g., ages 21–24). The mystery shoppers are old enough to purchase and consume alcohol legally, but young enough to trigger a request for ID. In some jurisdictions, mystery shops can be conducted legally with underage shoppers. Mystery shop programs typically provide owners and managers with written feedback on staff performance. They can also provide immediate feedback to the clerk or server, reinforcing appropriate ID-checking and, if staff fail the mystery shop, citing the need to check IDs for all young-appearing customers (e.g., 30 years or younger, depending on the outlet’s policy).

There are few systematic investigations of mystery shop programs. A case study that assessed a mystery shop program implemented in convenience stores near tribal reservations in Southern California found an increase in ID-checking from 67% at baseline to 100% three months after implementation [31]. The lack of a control group, however, makes interpretation of these results difficult. Positive results were also found in a multisite randomized trial in 36 Oregon communities that tested a comprehensive intervention that included a mystery shop program, but also enforcement compliance checks, media, and other community strategies [32]. ID-checking increased by 18% in intervention communities compared with 9% in control communities. A tobacco mystery shop program was a part of a broad-based community program implemented in two sets of cities to decrease youth smoking prevalence [33, 34]. Substantial decreases in sales rates were observed in the first (64% vs. 24%) and second (67% vs. 22%) sets of communities, respectively. Because the mystery shop interventions in these latter studies were embedded within broader community prevention efforts, it is difficult to attribute reductions in sales to the mystery shops alone.

In one of the few randomized studies evaluating alcohol mystery shop feedback, shop owners in the Netherlands received letters that summarized the results of underage mystery shops and provided information about age restrictions on alcohol sales [35]. Sales refusal rates among the outlets in the experimental condition increased from 18% prior to the intervention to 32% post intervention. A non-significant decrease in sales refusals was observed in the control condition (19% vs. 9%). Interestingly, outlets in both the experimental and control conditions showed increased rates of ID-checking, but these increases did not translate into improved sales compliance among shops in the control condition. Although this study demonstrates the potential of mystery shop interventions, it was limited to a single pre-test and single post-test observation. The extent to which the effects could be sustained or improved by providing ongoing feedback was not addressed.

Finally, a smaller randomized study investigated the effect of a mystery shop program on improving tobacco ID-checking by clerks for a major pharmacy chain that sells cigarettes [36]. Stores receiving feedback showed significant increases in ID-checking rates relative to controls, although the control stores showed gradual improvement and caught up with the intervention stores by the end of the program, suggesting that this study may have suffered from cross-contamination due to store managers informing their colleagues that mystery shops were underway.

Although these studies are suggestive, no published randomized multisite trial has investigated whether a responsible alcohol retailing program that features ongoing mystery shops, separate from other community interventions, can improve ID-checking performance. The present study fills this gap. We hypothesized that providing mystery shop feedback to clerks, servers, and owners/managers of on- and off-premises alcohol outlets would significantly improve ID-checking for alcoholic beverage sales. Because policies and rates of underage sales differ between on-premises and off-premises outlets [37], as a secondary research question we investigated whether mystery shop feedback differentially affected ID-checking at on- and off-premises establishments, but did not have a specific hypothesis.

Method

Study design

Because cross-contamination among outlets is a potential problem [36], the study used a cluster randomized cross-over design implemented with 557 on- and off-premises alcohol outlets located in 16 communities in four states in the U.S. (California, Massachusetts, Texas, and Wisconsin). Based on pilot data, and taking design effects into account, the numbers of communities and outlets were selected to provide 80% power to detect medium effect sizes. Following the first three monthly mystery shops, communities were randomly assigned to an Early Intervention condition (n = 8) or to a Delayed Intervention condition (n = 8) using a computerized random selection procedure.

In the Early Intervention group, feedback was provided immediately after the fourth mystery shop and continued for five additional mystery shops, after which these establishments were retired from the study. For the Delayed Intervention group, feedback began immediately after a tenth mystery shop attempt and continued for five additional monthly mystery shops. Because feedback was not provided until after the 4th or 10th purchase attempts were completed, these mystery shops served as the final pre-intervention baseline measures for the Early and Delayed Intervention groups, respectively. Measures of staff ID-checking behaviour were obtained from all outlets during the baseline and early intervention periods, with the Delayed Intervention group serving as a control for the Early Intervention group. During the final phase of the project, these measures were obtained only from outlets in the Delayed Intervention communities.

The cross-over design was selected, in part, based on ethical considerations to allow outlets initially in the control condition to receive the benefits of mystery shop performance feedback. In addition, this design allowed us to replicate the findings, thus adding to our confidence in them.

Community sample

Working with each state’s alcohol regulatory agency and retail association partners, we identified two pairs (four communities) in each state with populations between 50,000–250,000 that had sufficient numbers of licensees for the study. The 16 selected communities were diverse in terms of race/ethnicity and social-economic status and were selected to represent different geographical areas and demographic profiles within each state. Each pair of communities was matched as closely as possible based on population, numbers of retail outlets, median family income, percentage of population under age 18 years, and racial/ethnic composition (e.g., percentage White) using Euclidean distance scores calculated using a computerized algorithm. To avoid cross-community contamination, we required that the communities not be contiguous with one another nor adjacent to any community that was contiguous with a sampled community.

Outlet sample

Based on state alcohol licensing data we randomly sampled 324 on-premises serving establishments and 313 off-premises retailers to be invited into the study (N = 637). On-premises licensees included bars, taverns, clubs, and restaurants that received 40% or more of their revenues from liquor sales. Restaurants where the sale of alcohol was incidental and therefore unlikely venues for underage drinking were excluded. Off-premises licensees included liquor stores and other retailers (e.g., grocery stores, convenience stores, drug stores) that sold alcohol for consumption elsewhere.

Participation rates

Out of the 637 licensed outlets sampled for the study, 29 (5%) were found to be out of business or going out of business. In addition, 17 (3%) were determined to be ineligible because they: (a) were not open to the public (n = 2); (b) were not open during evening hours when the mystery shops would be conducted (n = 4); (c) did not sell alcohol (n = 1); (d) were a family restaurant (n = 1); (e) required a cover charge (n = 1); (f) were a concert hall, high-end speciality shop, or auction house (n = 3); or (g) were already part of a commercial program providing mystery shop feedback (n = 5). Of the 591 eligible outlets, 557 (94%) participated in the study. Participation rates were similar for off-premises (96%) and on-premises (93%) establishments. On average, 17 (range = 14–20) on-premises and 18 (range = 11–23) off-premises outlets participated in each community.

Recruitment

The selected licensees in each community received three letters before the intervention began. The first two letters described the study, noted that participation was voluntary, and asked the licensees to contact the research coordinator if they had any questions or concerns. The third and final letter served as a reminder that the program would begin soon and gave information about what to expect.

Intervention

Mystery shop inspections

The mystery shop inspections involved attempts by young, but legal-age (21–24 years old) customers to purchase alcohol in order to observe whether staff requested an ID prior to making a sale. Although the mystery shoppers were of legal age, they were young enough to trigger a request for ID. The mystery shop inspections were conducted by the BARS Program, which recruits and trains mystery shoppers for alcohol purchase attempts (http://www.barsprogram.com/).

Feedback

The staff received immediate feedback during the intervention. When servers or clerks appropriately requested an ID, the shopper gave them a Green Card explaining that they had passed an inspection. If staff did not ask for ID, the shopper immediately presented them with a Red Card explaining that they had failed an inspection. In addition to the immediate feedback to the clerk or server, the licensees received a written follow-up report indicating that their establishment had been inspected and noting whether an ID had been requested.

Other intervention materials

In addition to tailored reports, licensees received (a) a summary of their state’s sales-to-minor laws and a set of recommended alcohol sales policies (e.g., check ID for anyone appearing to be under age 30), which they could modify to reflect their own policies and preferred wording; (b) a monthly report showing the aggregate mystery shop results for their community; and (c) other supporting materials to help licensees work more effectively with staff (e.g., How to Talk to Employees about Mystery Shop Inspections, Employee Pledge, The Case for Responsible Retailing). During the intervention periods, advertisements were placed in local newspapers to let the community know that the program was underway, although the participating retailers were not named.

Outcome Measure

In addition to being a central component of the intervention, the monthly mystery shop inspections provided a measure of staff age-verification behaviour that served as the study outcome. After each attempt, the mystery shoppers recorded whether age verification had been requested (0 = failed to request ID; 1 = requested ID). Clerks and servers were blind to whether a customer was a mystery shopper, whose identity was not revealed until after the purchase attempt was completed. To prevent clerks and servers from becoming familiar with the mystery shoppers, a given shopper never visited the same establishment within two consecutive months, and the days and times of the visits for each establishment were varied. As a result, the mystery shops provide an unbiased assessment of age-verification behaviour.

Research Involving Human Participants

The study was reviewed and approved by the Institutional Review Board (IRB) of the Pacific Institute for Research and Evaluation. The study was conducted in accordance with the ethical standards of the 1964 Declaration of Helsinki and its later amendments. The study did not qualify as a clinical trial under the National Institute of Health’s guidelines at the time of funding and registration was not required.

Analyses

To account for nesting of observations, the analyses comprised three-level fixed-effects hierarchical logistic regressions with repeated mystery shop observations (Level I) nested within the 557 outlets (Level II; ICC = .08), nested within the 16 communities (Level III; ICC = .06). A time-varying dummy variable representing the intervention was included at Level I. This variable was coded as 0 for both the Early and Delayed Intervention groups during the baseline when no feedback was provided, as 1 for the Early Intervention group and 0 for the Delayed Intervention group during the initial intervention period, and as 1 for the Delayed Intervention group during the delayed intervention period. That is, the intervention variable was coded as 1 during periods when outlets were exposed to the intervention and as 0 otherwise. Because outlet policies and rates of underage sales differ for on- and off-premises establishments [37], outlet type (on- versus off-premises) was included as a covariate at Level II (0 = on-premises). No community variables were included as covariates at Level III, although nesting within communities was accounted for in the analyses. Additional analyses explored whether mystery shop feedback differentially affected ID-checking at on- and off-premises outlets. An intent-to-treat approach was used with all outlets included in the analyses to avoid biases that can result from per protocol analyses [38].

Results

ID-checking rates

Figure 1 shows the combined ID-checking rates for on- and off-premises outlets in the Early and Delayed Intervention groups across the three phases of the study. Overall, the intervention was associated with a two-fold increase in the odds of checking IDs, OR = 2.05; t (537) = 9.22; p < .001; 95% CI = 1.76, 2.39. The ID-checking rate in the Early Intervention group increased from an average of 80% during the baseline to an average of 90% during the initial intervention period, achieving 96% by the end of the that phase. For the Delayed Intervention group, the rate of checking IDs increased from an average of 80% pre-intervention to 88% during the delayed intervention period, reaching 94% by the end of the study.

Figure 1.

Figure 1

Open in a new tab

ID-checking pass rates for all outlets receiving early and delayed Mystery Shop feedback interventions.

ID-checking rates among on-premises and off-premises outlets

Overall, off-premises outlets were somewhat less likely to check IDs (83%) than were on-premises outlets (85%), OR = 0.84; t (537) = 2.03; p < 0.05; 95% CI = 0.71, 0.99. Both types of outlets, however, increased ID-checking rates during the intervention periods, each achieving 93%–96% pass rates by the end of the program (Figure 2). The cross-level outlet type by intervention interaction was not significant, OR = 1.27; t (537) = 1.48; p < 0.14; 95% CI = 0.92, 1.74, indicating that the mystery shop feedback had a similar effect on ID-checking at both types of outlets. Separate analyses by outlet type confirmed that the intervention significantly increased ID-checking for both on-premises outlets, OR = 1.79; p < .001; 95% CI =1.43, 2.24, and off-premises outlets, OR = 2.29; p < .001; 95% CI = 1.86, 2.82.

Figure 2.

Figure 2

Open in a new tab

ID-checking pass rates for on-premises and off-premises outlets receiving early and delayed Mystery Shop feedback interventions.

Discussion

This study provides evidence that a responsible retailing program employing young, legal-age mystery shoppers who attempt to purchase alcohol and then provide feedback on whether the clerk or server asks for an ID can improve ID-checking performance. The intervention doubled the odds of checking IDs, overall achieving 94%–96% pass rates by the end of the intervention. Importantly, the intervention was similarly effective at both on- and off-premises alcohol establishments.

A major strength of the study is its design: a multisite randomized trial conducted in 16 communities in four U.S. states with a delayed intervention control group that also showed positive program effects once the intervention was introduced. Importantly, the participation rate among the randomly selected licensees was high (94%), suggesting that the likelihood of selection bias was minimal and that mystery shop programs are broadly acceptable to alcohol retailers. This study thus provides the strongest evidence to date that a mystery shop program, independently of other community interventions, can improve staff age-verification behaviour. Future controlled studies are needed to investigate whether this type of responsible retailing program can improve ID-checking to prevent sales of underage tobacco and other age-restricted products.

The study does have limitations that should be acknowledged. First, we did not follow-up with the unreported mystery shops conducted with the early intervention group during the delayed-intervention phase. Thus, we are unable to determine how quickly the effects of the mystery shops might have decayed once the intervention ended. Some evidence suggests that the effects of enforcement compliance checks decay after about three months [22], and it is possible that a similar outcome might result for mystery shop interventions. An advantage of mystery shop programs, however, is that they can be implemented on an ongoing basis by local public and private stakeholders, including non-profit agencies, prevention coalitions, regulators, or distributors and producers at significantly less cost than compliance checks. As a result, mystery shop programs may be easier to maintain over time. The reinforcing effect of sponsorship by local stakeholders may also increase program impact.

Second, the study was conducted in 16 midsized communities in four states of the United States. It is unclear if the intervention would be as effective in other communities and states or outside the U.S., especially where alcohol controls are weak [39].

There are additional research questions that remain. First, the mystery shops were conducted on a monthly basis, which may not be sustainable long-term even with the involvement of local stakeholders. A future investigation should explore whether bimonthly or quarterly visits are as effective since previous small-scale studies have provided conflicting evidence [36, 40]. Yet another option would be to begin with monthly shops, but then move to a random schedule of less frequent shops once high age-verification rates are achieved. Second, the tailored feedback provided to licensees was reinforced by an additional monthly report that provided the aggregate mystery shop results for their community. Whether these aggregate reports alone, serving as a reminder to licensees, can motivate improved age-verification behaviour also deserves study. Third, our study only investigated the effects of a mystery shop program on ID-checking and not on actual sales. Although requesting an ID is predictive of refusing sales to minors [9, 15, 41], clerks and servers may nonetheless complete sales of age-restricted products even when IDs are checked. In part, we used this protocol because of legal issues in many U.S. states relating to the use minors to attempt purchases of alcohol other than as a part of official enforcement compliance checks. Future research should investigate the effects of mystery shop programs not only on ID-checking, but also on sales. Fourth, methods for encouraging high participation rates, such as insurance incentive programs or mitigated penalties for violating law enforcement compliance checks, should be explored.

Finally, we investigated clerk and server ID-checking behaviour, but did not examine whether the mystery shop program impacted young people’s access to alcohol or their drinking behaviours. The intervention involved a limited number of licensees in each community, thus leaving non-participating licensees as potential targets for underage youth seeking to purchase alcohol. Whether mystery shop programs can reduce underage drinking or deter attempts by underage youth to purchase alcohol remains to be seen. Whatever deterrence effect a mystery shop intervention might have on youth may depend on having a high-profile communications campaign to inform them about the program and its impact on retailer vigilance [42], thereby increasing their perceived likelihood of being sanctioned or refused service.

From the perspective of regulators, police, prevention advocates, and the retailers themselves, compliance with underage sales laws is an important goal. Responsible retailing programs that use mystery shops to provide ongoing feedback about staff performance can assist retailers in complying with the law and play an important role as part of a comprehensive alcohol control system that includes enforcement and sanctions for retailers who sell alcohol to minors.

Acknowledgments

This study and preparation of this manuscript were supported by grant number R44AA021593 from the National Institute on Alcohol Abuse and Alcoholism (NIAAA) of the U.S. National Institutes of Health (NIH). Additional support for Dr. Grube and Dr. Lipperman-Kreda during the preparation of this manuscript was provided by NIAAA grant number P60-AA006282. The content is solely the responsibility of the authors and does not necessarily represent the official views of NIAAA or NIH.

Disclosure of possible conflicts of interest. The program evaluated in this study was developed by the Responsible Retailing Forum (RRF), a 501c(3) non-profit corporation. Dr. Krevor is President of RRF and also President of RRF Field Services LLC, the research and technical services arm of RRF. Ms. DeJong is Director of Field Services for RRF Field Services LLC. Drs. DeJong and Grube are Members of the Board of Directors for RRF. Drs. Krevor, Grube, and DeJong have been supported within the past three years by funding from the alcohol industry to evaluate industry-sponsored programmes to reduce alcohol sales to minors and other alcohol-related harms. Dr. Lipperman-Kreda has no conflicts of interest to declare.

Contributor Information

Joel W. Grube, Prevention Research Center, Pacific Institute for Research and Evaluation

William DeJong, School of Public Health, Boston University

Maureen DeJong, Responsible Retailing Forum

Sharon Lipperman-Kreda, Prevention Research Center, Pacific Institute for Research and Evaluation

Brad S. Krevor, Responsible Retailing Forum

References

  • 1.WHO. Global Status Report on Alcohol and Health 2014. Geneva, Switzerland: World Health Organization; 2014. [Google Scholar]
  • 2.DeJong W, Blanchette J. Case closed: research evidence on the positive public health impact of the age 21 minimum legal drinking age in the United States. Journal of studies on alcohol and drugs Supplement. 2014;75(Suppl 17):108–15. doi: 10.15288/jsads.2014.s17.108. [DOI] [PubMed] [Google Scholar]
  • 3.Gruenewald PJ. Commentary on: The Impact of the Minimum Legal Drinking Age on Alcohol-Related Chronic Disease Mortality. Alcoholism, clinical and experimental research. 2016;40(9):1822–4. doi: 10.1111/acer.13161. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 4.ESPAD Report 2015. Results from the European School Survey Project on Alcohol and Other Drugs. Lisbon, Portugal: EMCDDA/ESPAD; 2016. Available from: http://espad.org/sites/espad.org/files/TD0116475ENN.pdf. [Google Scholar]
  • 5.Miech RA, Johnston LD, O’Malley PM, Bachman JG, Schulenberg JE, Patrick ME. Monitoring the Future national survey results on drug use, 1975–2016: Volume I, Secondary school students. Ann Arbor, MI: University of Michigan; 2017. [Google Scholar]
  • 6.Freisthler B, Gruenewald PJ, Treno AJ, Lee J. Evaluating alcohol access and the alcohol environment in neighborhood areas. Alcoholism, clinical and experimental research. 2003;27(3):477–84. doi: 10.1097/01.ALC.0000057043.04199.B7. [DOI] [PubMed] [Google Scholar]
  • 7.Huckle T, Conway K, Casswell S, Pledger M. Evaluation of a regional community action intervention in New Zealand to improve age checks for young people purchasing alcohol. Health promotion international. 2005;20(2):147–55. doi: 10.1093/heapro/dah610. [DOI] [PubMed] [Google Scholar]
  • 8.Lynne-Landsman SD, Kominsky TK, Livingston MD, Wagenaar AC, Komro KA. Alcohol Sales to Youth: Data from Rural Communities Within the Cherokee Nation. Prevention science : the official journal of the Society for Prevention Research. 2016;17(1):32–9. doi: 10.1007/s11121-015-0579-2. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 9.Paschall MJ, Grube JW, Black C, Flewelling RL, Ringwalt CL, Biglan A. Alcohol outlet characteristics and alcohol sales to youth: results of alcohol purchase surveys in 45 Oregon communities. Prevention science : the official journal of the Society for Prevention Research. 2007;8(2):153–9. doi: 10.1007/s11121-006-0063-0. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 10.Rowland BC, Hall JK, Kremer PJ, Miller PG, Toumbourou JW. Underage purchasing of alcohol from packaged liquor outlets: an Australian study. Health promotion international. 2017;32(5):790–9. doi: 10.1093/heapro/daw007. [DOI] [PubMed] [Google Scholar]
  • 11.Toomey TL, Komro KA, Oakes JM, Lenk KM. Propensity for illegal alcohol sales to underage youth in Chicago. Journal of community health. 2008;33(3):134–8. doi: 10.1007/s10900-007-9080-6. [DOI] [PubMed] [Google Scholar]
  • 12.van Hoof JJ, Gosselt JF. Underage alcohol sales--it only takes a minute: a new approach to underage alcohol availability. Journal of studies on alcohol and drugs. 2013;74(3):423–7. doi: 10.15288/jsad.2013.74.423. [DOI] [PubMed] [Google Scholar]
  • 13.Van Hoof JJ, Roodbeen RT, Krokke J, Gosselt JF, Schelleman-Offermans K. Alcohol sales to underage buyers in the Netherlands in 2011 and 2013. The Journal of adolescent health : official publication of the Society for Adolescent Medicine. 2015;56(4):468–70. doi: 10.1016/j.jadohealth.2014.11.025. [DOI] [PubMed] [Google Scholar]
  • 14.Willner P, Hart K, Binmore J, Cavendish M, Dunphy E. Alcohol sales to underage adolescents: an unobtrusive observational field study and evaluation of a police intervention. Addiction (Abingdon, England) 2000;95(9):1373–88. doi: 10.1046/j.1360-0443.2000.95913738.x. [DOI] [PubMed] [Google Scholar]
  • 15.Gosselt JF, van Hoof JJ, de Jong MDT, Prinsen S. Mystery Shopping and Alcohol Sales: Do Supermarkets and Liquor Stores Sell Alcohol to Underage Customers? Journal of Adolescent Health. 2007;41(3):302–8. doi: 10.1016/j.jadohealth.2007.04.007. [DOI] [PubMed] [Google Scholar]
  • 16.Federal Trade Commission. Alcohol Marketing and Advertising: A Report to Congress. Washington, DC: FTC; 2003. [Google Scholar]
  • 17.Bonnie RJ, O’Connell ME, editors. National Research Council and Institute of Medicine, Drinking CoDaStRaPU. Reducing underage drinking: A collective responsibility. Washington, DC: The National Academies Press; 2004. [PubMed] [Google Scholar]
  • 18.Office of the Surgeon General. The Surgeon General’s Call to Action to Prevent and Reduce Underage Drinking. Washington, DC: 2007. [PubMed] [Google Scholar]
  • 19.Erickson DJ, Smolenski DJ, Toomey TL, Carlin BP, Wagenaar AC. Do alcohol compliance checks decrease underage sales at neighboring establishments? Journal of studies on alcohol and drugs. 2013;74(6):852–8. doi: 10.15288/jsad.2013.74.852. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 20.Grube JW. Preventing sales of alcohol to minors: results from a community trial. Addiction (Abingdon, England) 1997;92(Suppl 2):S251–60. [PubMed] [Google Scholar]
  • 21.Scribner R, Cohen D. The effect of enforcement on merchant compliance with the minimum legal drinking age law. Journal of Drug Issues. 2001;31(4):857–66. [Google Scholar]
  • 22.Wagenaar AC, Toomey TL, Erickson DJ. Preventing youth access to alcohol: outcomes from a multi-community time-series trial*. Addiction (Abingdon, England) 2005;100(3):335–45. doi: 10.1111/j.1360-0443.2005.00973.x. [DOI] [PubMed] [Google Scholar]
  • 23.Elder RW, Lawrence BA, Janes G, Brewer RD, Toomey TL, Hingson RW, et al. Enhanced enforcement of laws prohibiting sale of alcohol to minors: systematic review of effectiveness for reducing sales and underage drinking. Transportation Research Circular. 2007;E-C123:181–8. [Google Scholar]
  • 24.Erickson DJ, Lenk KM, Sanem JR, Nelson TF, Jones-Webb R, Toomey TL. Current use of underage alcohol compliance checks by enforcement agencies in the United States. Alcoholism, clinical and experimental research. 2014;38(6):1712–9. doi: 10.1111/acer.12397. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 25.Erickson DJ, Rutledge PC, Lenk KM, Nelson TF, Jones-Webb R, Toomey TL. Patterns of Alcohol Policy Enforcement Activities among Local Law Enforcement Agencies: A Latent Class Analysis. The international journal of alcohol and drug research. 2015;4(2):103–11. doi: 10.7895/ijadr.v4i2.204. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 26.Chinman M, Ebener P, Burkhart Q, Osilla KC, Imm P, Paddock SM, et al. Evaluating the impact of getting to outcomes-underage drinking on prevention capacity and alcohol merchant attitudes and selling behaviors. Prevention science : the official journal of the Society for Prevention Research. 2014;15(4):485–96. doi: 10.1007/s11121-013-0389-3. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 27.Toomey TL, Erickson DJ, Lenk KM, Kilian GR, Perry CL, Wagenaar AC. A randomized trial to evaluate a management training program to prevent illegal alcohol sales. Addiction (Abingdon, England) 2008;103(3):405–13. doi: 10.1111/j.1360-0443.2007.02077.x. discussion 14-5. [DOI] [PubMed] [Google Scholar]
  • 28.Roodbeen RT, Schelleman-Offermans K, Lemmens PH. Alcohol and Tobacco Sales to Underage Buyers in Dutch Supermarkets: Can the Use of Age Verification Systems Increase Seller's Compliance? The Journal of adolescent health : official publication of the Society for Adolescent Medicine. 2016;58(6):672–8. doi: 10.1016/j.jadohealth.2016.03.005. [DOI] [PubMed] [Google Scholar]
  • 29.Krevor BS, Capitman JA, Oblak L, Cannon JB, Ruwe M. Preventing illegal tobacco and alcohol sales to minors through electronic age-verification devices: a field effectiveness study. J Public Health Policy. 2003;24(3–4):251–68. [PubMed] [Google Scholar]
  • 30.Van Hoof JJ. The effectiveness of ID readers and remote age verification in enhancing compliance with the legal age limit for alcohol. European journal of public health. 2017;27(2):357–9. doi: 10.1093/eurpub/ckw183. [DOI] [PubMed] [Google Scholar]
  • 31.Moore RS, Roberts J, McGaffigan R, Calac D, Grube JW, Gilder DA, et al. Implementing a reward and reminder underage drinking prevention program in convenience stores near Southern California American Indian reservations. The American journal of drug and alcohol abuse. 2012;38(5):456–60. doi: 10.3109/00952990.2012.696758. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 32.Flewelling RL, Grube JW, Paschall MJ, Biglan A, Kraft A, Black C, et al. Reducing youth access to alcohol: findings from a community-based randomized trial. American journal of community psychology. 2013;51(1–2):264–77. doi: 10.1007/s10464-012-9529-3. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 33.Biglan A, Ary D, Koehn V, Levings D, Smith S, Wright Z, et al. Mobilizing positive reinforcement in communities to reduce youth access to tobacco. American journal of community psychology. 1996;24(5):625–38. doi: 10.1007/BF02509717. [DOI] [PubMed] [Google Scholar]
  • 34.Biglan A, Ary DV, Smolkowski K, Duncan T, Black C. A randomised controlled trial of a community intervention to prevent adolescent tobacco use. Tobacco control. 2000;9(1):24–32. doi: 10.1136/tc.9.1.24. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 35.Van Hoof JJ, Gosselt JF, Baas N, De Jong MD. Improving shop floor compliance with age restrictions for alcohol sales: effectiveness of a feedback letter intervention. European journal of public health. 2012;22(5):737–42. doi: 10.1093/eurpub/ckr162. [DOI] [PubMed] [Google Scholar]
  • 36.Krevor BS, Ponicki WR, Grube JW, DeJong W. The effect of mystery shopper reports on age verification for tobacco purchases. Journal of health communication. 2011;16(8):820–30. doi: 10.1080/10810730.2011.561912. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 37.Chinman M, Burkhart Q, Ebener P, Fan CC, Imm P, Osilla KC, et al. The premises is the premise: understanding off- and on-premises alcohol sales outlets to improve environmental alcohol prevention strategies. Prevention science : the official journal of the Society for Prevention Research. 2011;12(2):181–91. doi: 10.1007/s11121-011-0203-z. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 38.Mansournia MA, Higgins JP, Sterne JA, Hernan MA. Biases in Randomized Trials: A Conversation Between Trialists and Epidemiologists. Epidemiology (Cambridge, Mass) 2017;28(1):54–9. doi: 10.1097/EDE.0000000000000564. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 39.Dorji G, DeJong W, Bor J, Bachman DeSilva M, Sabin L, Feeley FR, et al. Increasing compliance with alcohol service laws in a developing country: intervention trial in the Kingdom of Bhutan. Addiction (Abingdon, England) 2016;111(3):467–74. doi: 10.1111/add.13202. [DOI] [PubMed] [Google Scholar]
  • 40.Krevor BS, DeJong W, DeJong MA, Grube JW. Impact of mystery shopper reports on age verification for alcohol purchases: The Respect 21TM Responsible Retaining Forum. Waltham, MA: The Responsible Retailing Forum; 2017. [Google Scholar]
  • 41.Rossow I, Karlsson T, Raitasalo K. Old enough for a beer? Compliance with minimum legal age for alcohol purchases in monopoly and other off-premise outlets in Finland and Norway. Addiction (Abingdon, England) 2008;103(9):1468–73. doi: 10.1111/j.1360-0443.2008.02302.x. [DOI] [PubMed] [Google Scholar]
  • 42.DeJong W, Hingson R. Strategies to reduce driving under the influence of alcohol. Annual Review of Public Health. 1998;19:359–78. doi: 10.1146/annurev.publhealth.19.1.359. [DOI] [PubMed] [Google Scholar]

RESOURCES