Abstract
This quality improvement study assesses the potential consequences of systematic underreporting of alcohol use on the observed results in a recent, well-publicized study.
It is well appreciated that systematic measurement error can bias estimates of the population mean (eg, biased estimates of average alcohol use), but the consequences of systematic measurement error on the associations between risk factors and outcomes are often overlooked. A recent example is a study1 that led to widely publicized reports of no safe level of alcohol consumption. The study did not account for potential misclassification owing to underreporting of alcohol use. Systematic underreporting of alcohol use could result in overestimating the association between a low amount of alcohol consumption and the risk of adverse health outcomes, especially given evidence that heavy drinkers underreport their true levels of alcohol consumption by up to 40% to 65%.2 Therefore, this quality improvement study assessed the potential implications of systematic underreporting of alcohol use on the observed results in that recent, well-publicized study.1
Methods
In this quality improvement study, we conducted a sensitivity analysis between November 2018 and February 2019 to assess the degree of misclassification required to result in a substantive bias in estimating the risk of low levels of alcohol consumption. The following 3 key parameters were set to vary: (1) the distribution of drinkers in the population, (2) the true relative risk of adverse health outcomes attributable to alcohol use at each level of drinking, and (3) the percentage of drinkers who underreport their alcohol use. We used pooled results from 3 major studies3,4,5 on drinking prevalence to derive the distribution of drinks per day in the study population (Table). The true relative risk of cardiac or all-cause mortality over a period of years associated with alcohol consumption was assumed to have an underlying J-shaped relative risk curve (positive health consequences at low levels of alcohol use) and was derived from the smoothed distribution of relative risks reported in the original study1 and one other major report.6 The percentage of drinkers who underreport their alcohol use was set to vary from 0% to the percentage of underreporting needed to obscure the underlying J-shaped curve and reproduce the observed results in the original study.1 At-risk drinking was defined as 3 to 5 drinks per day, and heavy drinking was defined as 6 or more drinks per day. The following 2 types of underreporting were assumed: (1) underreporting by 1 to 2 drinks per day for at-risk drinkers and (2) dramatic underreporting by heavy drinkers who were trying to hide a drinking problem (eg, reporting 1-2 drinks per day when true drinking was ≥6 drinks per day). This study did not require submission to the University of Michigan institutional review board per institutional policy.
Table. Relative Risk (RR) of Adverse Health Outcomes Due to Alcohol Consumption, Varied by Percentage Underreporting Compared With Hypothetical True RR and RR Observed in the Global Burden of Disease Study1.
| No. of Drinks per Day (% of Population by Self-report)a | Assumed True RR | Observed RR if 20% of ≥3 Drinks per Day Drinkers Underreportb | Observed RR if 30% of ≥3 Drinks per Day Drinkers Underreportc | RR Observed in Original Studyd |
|---|---|---|---|---|
| 0 (16.5) | 1.000 | 1.000 | 1.000 | 1.000 |
| 1 (40.0) | 0.975 | 0.999 | 1.017 | 1.010 |
| 2 (20.0) | 1.000 | 1.055 | 1.088 | 1.100 |
| 3 (9.0) | 1.190 | 1.198 | 1.203 | 1.200 |
| 4 (5.0) | 1.230 | 1.236 | 1.242 | 1.250 |
| 5 (2.5) | 1.340 | 1.340 | 1.340 | 1.340 |
| 6 (2.0) | 1.500 | 1.500 | 1.500 | 1.500 |
| 7 (1.5) | 1.600 | 1.600 | 1.600 | 1.600 |
| 8 (1.0) | 1.850 | 1.850 | 1.850 | 1.850 |
| 9 (0.5) | 2.000 | 2.000 | 2.000 | 2.000 |
| ≥10 (2.0) | 2.350 | 2.350 | 2.350 | 2.350 |
The proportion of the population in each drinking category is estimated based on smoothed estimates of 3 studies.3,4,5
Given the parameters in the first 2 columns, these results are observed if (1) 10% of those drinking 3 to 5 drinks per day underreported by 1 drink per day, (2) 10% of those drinking 3 to 5 drinks per day underreported by 2 drinks per day, (3) 10% of those drinking 6 or more drinks per day only reported 1 drink per day, and (4) 10% of those drinking 6 or more drinks per day only reported 2 drinks per day.
Given the parameters in the first 2 columns, these results are observed if (1) 15% of those drinking 3 to 5 drinks per day underreported by 1 drink per day, (2) 15% of those drinking 3 to 5 drinks per day underreported by 2 drinks per day, (3) 15% of those drinking 6 or more drinks per day only reported 1 drink per day, and (4) 15% of those drinking 6 or more drinks per day only reported 2 drinks per day.
Estimates taken from Figure 5 of the Global Burden of Disease Study.1
Results
With 30% of those drinking at least 3 drinks per day underreporting, the observed relative risks of those reporting 1 to 2 drinks per day approximated the observed risks in the original study1 that reported harm from low levels of alcohol consumption (Table). With 40% of at-risk and heavy drinkers underreporting, the observed relative risks began to exceed some of the harm estimates reported in the original study.1
Discussion
If 30% of at-risk drinkers underreport their drinking by 1 to 2 drinks per day and if 30% of heavy drinkers try to hide their drinking by saying they only drink 1 to 2 drinks per day, this underreporting could lead to biased study findings that misrepresent a reality in which 1 drink per day reduces the risk of adverse health outcomes by 5% and 2 drinks per day result in no net harm.
We did not examine biases related to misclassification of nondrinkers vs drinkers, which would generally bias results toward the null. All 3 of the parameters we varied—distribution of drinkers, true risk of adverse health outcomes due to alcohol use, and percentage of drinkers underreporting—are not known precisely, and the parameters we used are rough estimates based on available literature. Our intent was to show a potential problem with reported risk estimates, not to generate new risk estimates that can reliably be used in real-world settings.
The present analysis demonstrates the importance of considering how systematic underestimation of a risk factor can overestimate the consequences of low exposure to the risk factor. Furthermore, strong conclusions are always risky when the observed effect sizes are small. If researchers have a good reason to suspect systematic error in risk factor assessment, a sensitivity analysis should be conducted to assess the implications of this error on observed associations between risk factors and outcomes.
References
- 1.GBD 2016 Alcohol Collaborators Alcohol use and burden for 195 countries and territories, 1990-2016: a systematic analysis for the Global Burden of Disease Study 2016 [published corrections appear in Lancet. 2018;392(10153):1116 and 2019;393(10190):e44]. Lancet. 2018;392(10152):1015-1035. doi: 10.1016/S0140-6736(18)31310-2 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.Stockwell T, Zhao J, Macdonald S. Who under-reports their alcohol consumption in telephone surveys and by how much? an application of the “yesterday method” in a national Canadian substance use survey. Addiction. 2014;109(10):1657-1666. doi: 10.1111/add.12609 [DOI] [PubMed] [Google Scholar]
- 3.Mukamal KJ, Conigrave KM, Mittleman MA, et al. Roles of drinking pattern and type of alcohol consumed in coronary heart disease in men. N Engl J Med. 2003;348(2):109-118. doi: 10.1056/NEJMoa022095 [DOI] [PubMed] [Google Scholar]
- 4.Askgaard G, Grønbæk M, Kjær MS, Tjønneland A, Tolstrup JS. Alcohol drinking pattern and risk of alcoholic liver cirrhosis: a prospective cohort study. J Hepatol. 2015;62(5):1061-1067. doi: 10.1016/j.jhep.2014.12.005 [DOI] [PubMed] [Google Scholar]
- 5.Bobak M, Malyutina S, Horvat P, et al. Alcohol, drinking pattern and all-cause, cardiovascular and alcohol-related mortality in Eastern Europe. Eur J Epidemiol. 2016;31(1):21-30. doi: 10.1007/s10654-015-0092-8 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Gmel G, Gutjahr E, Rehm J. How stable is the risk curve between alcohol and all-cause mortality and what factors influence the shape? a precision-weighted hierarchical meta-analysis. Eur J Epidemiol. 2003;18(7):631-642. doi: 10.1023/A:1024805021504 [DOI] [PubMed] [Google Scholar]