A systematic review and meta-analysis of alcohol consumption and 1 injury risk as a function of study design and recall period

Abstract

Background

It is well established that alcohol consumption is associated with an increased risk of injury. This systematic review and meta-analysis addresses important methodological issues commonly encountered in the alcohol and injury field by delineating the effect of study design and alcohol consumption recall period on effects size magnitude and by conducting gender-specific analyses.

Methods

Meta-analyses using random effect models. Data sources were peer-reviewed studies on alcohol and injury from 1970 to 2009 from MEDLINE, Psych Info and on-line journals. Case-control or case crossover ED studies reporting injury risk and alcohol consumed six hours before injury were included.

Results

The overall odds of injury were 2.799 (2.214–3.538, p<0.001). For case-crossover studies, the odds were 3.815 (2.646–5.499, p<0.001), for ED case-control studies the odds were 1.977 (1.385–2.821, p<0.001) and for population case-control designs the odds were 3.145 (1.583–6.247, p<0.005). The ‘usual frequency’ recall period yielded an odds ratio of 4.235 (2.541–7.057, p<0.001), compared to 2.320 (1.789–3.008, p<0.001) for all other methods. There were significant differences in odds ratio magnitude when comparing studies by design and recall period. Females had higher odds of injury than males, 2.285 (1.361–3.836, p<0.005) vs. 1.071 (0.715–1.605, p=0.737).

Conclusions

Study design and alcohol consumption recall period have significant effects on effect size magnitude in estimating risk of injury from alcohol consumption six hours prior to injury. For the ‘usual frequency’ case-crossover design, significant moderator effects were found, resulting in over-estimates of injury risk from alcohol. ED case-crossover designs tend to overestimate risk and ED case-control designs tend to under-estimate. We provide recommendations for future ED research.

Introduction

It is well established that alcohol consumption is associated with an increased risk of injury (Rehm et al., 2003; Taylor et al., 2010). Literature has shown that alcohol consumption is associated with risk of unintentional and intentional injuries in cross-sectional (Goodman et al., 1991; Ivers and Ker, 2006; Malmivaara et al., 1993; Vingilis et al., 2007; Watt et al., 2004), case–crossover (Borges et al., 2006; Vinson et al., 2003), and case–control analyses (Peck et al., 2008). Given the opportunities they present to capture injured populations, emergency departments (EDs) have become a primary research focus. However, compared to autopsy studies or roadside surveys, ED studies have limitations with respect to deriving risk relations between alcohol use and injury. That is, injured patients drawn from ED samples are not representative of those who seek no treatment or those who receive treatment outside of an ED. Additionally, both injured and non-injured patients drawn from ED probability samples may not be representative of the ED patients from which they were drawn. Further, ED patients falling into such samples are more frequent users of EDs than those not falling into the sample; they may also differ on other important characteristics, such as drinking patterns and likelihood of injury compared to less frequent users (Cherpitel, 2007).

In the alcohol and injury ED literature, three main controlled study designs have been adopted:

1. ED case-control, where both the cases and the controls are drawn from patients attending the same EDs during the same period of time.

2. Population case-control, where the cases are drawn from ED patients and controls are drawn from the general population (i.e. not the ED setting); and,

3. Case-crossover, where the case also acts as his or her own control.

There is evidence that compared to population controls, non-injured ED control patients are more likely to be heavy or frequent drinkers and to experience ill-effects of alcohol (Cherpitel et al., 1992). Hence, it is reasonable to assume that ED studies which rely on the ED case-control designs are more likely to under-estimate the true magnitude of alcohol and injury associations than population case-control studies.

Population case-control studies are held to be a more methodologically sound approach to measuring associations between alcohol and injury than studies which use non-injured ED patient controls (Gmel and Daeppen, 2009). Population controls are usually matched to injured ED patients by age, sex, and location and are asked to report on drinking and other activities they were engaged in, sometimes at the same time as the injury occurred to their matched ‘case’. Population case-control studies do not suffer from the same potential bias inherent to the use of potentially alcohol-related ED-based controls because they sample from the general population falling within the catchment area of the ED. They are also less prone to the ill-effects of recall bias problems inherent to case-crossover studies.

In the case-crossover study, each case becomes his or her matched control. This is achieved by asking the injured patient (case) to recall alcohol consumption at the same time of day as the injury occurred the day before and/or the week before. Thus, drinking leading up to an injury event is compared to drinking undertaken at the same time on a previous day. An alternative method using the case-crossover design is the usual frequency method, which asks patients to report the typical quantity and typical frequency of drinking, usually over the last year. Based on this, one can estimate an expected likelihood of drinking at the time if there were no injury.

Methodological challenges of case-crossover studies

Case cross-over studies have been noted for their limitations regarding statistical ‘adjustment’ for usual consumption patterns (Maclure and Mittelman, 2000). This is because they effectively ignore those control-case pairs where drinking occurs at both times or where abstinence occurs at both times. One potential outcome is that in the case of the former, the sample becomes biased in favour of episodic drinkers, since regular drinkers who tend to consume alcohol at the same time of day are excluded. As episodic drinkers tend to have higher risks of injury than regular drinkers (Gmel et al., 2006; Hurst et al., 1994) such a sample bias would tend to inflate the apparent association between alcohol and injury. Further, it is inherent to case-crossover designs that the ‘control’ period is subject to greater recall bias than the information required to be recalled for the more recent case period. Specifically, recall bias is an issue when using the usual frequency approach to quantifying alcohol consumption. Gmel and Daeppen (2007) and Gmel (2010) warn that the usual frequency method tends to yield downward biased reports of alcohol consumption; when compared to case-control studies or case-crossover studies asking alcohol consumption yesterday, one obtains inaccurate effect size estimates. Studies of different self-report methods for assessing alcohol consumption have shown that usual frequency measures (e.g. typical quantity and typical frequency) substantially underestimate actual consumption in comparison with more recent recall (Stockwell et al., 2008), thus resulting in an overestimate of risk of injury related to drinking prior to the event.

Other shortcomings of the case-crossover design for estimating the magnitude of the alcohol-injury association include ‘spectrum’ bias whereby less severe injuries which occur during control periods are less likely to be recalled than the more serious injuries for which the patient sought ED treatment, uneven distribution of drinking over time; and unrepresentative control settings, such as context and individual characteristics(Bateson and Schwartz, 1999; Levy et al., 2001; Vines and Farrington, 2001). These biases particular to the case-crossover method tend to inflate risk estimates(Gmel and Daeppen, 2009).

Systematic reviews, meta-analyses and multicentre studies have been conducted recently on the relationship between alcohol consumption and injury (Borges et al., 2006; Taylor et al., 2010).Taylor et al. (2010) conducted a meta- analysis of 25 ED studies to quantify dose-response relationships between alcohol and injury. They reported that risk of injury increased non-linearly with increasing alcohol consumption and that study design had no significant effect.Borges et al. (2006) studied the risk of non-fatal injury at different levels of alcohol consumption in a multi-country study using a case crossover methodology. The authors found that the risk of injury increased with consumption of a single drink, and there was a 10-fold increase for patients who had consumed six or more drinks during the previous six hours. Many of the studies included in theTaylor et al. (2010) meta-analysis estimated level of alcohol consumption from measured blood alcohol concentrations (BAC). However, most of these studies did not take account of the length of time between injury and measurement of BAC and so may not be reliable or comparable to studies where injured individuals are asked to quantify their personal consumption in a fixed period leading up to the injury event.

There is some dispute in the literature as to whether injury risk differs between males and females. Case crossover studies find only small and insignificant differences (Borges et al., 2006) whereas population case-control studies have found very large differences in risks between males and females by level of consumption (e.g. McLeod et al., 1999).

The objective of this systematic review and meta-analysis was to address important methodological issues commonly encountered in the alcohol and injury field by delineating the effect of study design and alcohol consumption recall period on effects size magnitude and by conducting gender-specific analyses. Our meta-analysis fills current research gaps by examining risk relationships between alcohol consumption and injury, taking into account study design and recall period (usual frequency, ‘yesterday’ or ‘last week’) employed for comparison purposes, and investigating differences in risk for men and women. Both these issues are critical to accurately informing policy deliberations including estimating the risk of injury from alcohol consumption and the provision of advice on low-risk drinking for the general population. We test the hypotheses that (i) case-crossover studies create upward- biased relative risk estimates in comparison with case-control studies and (ii) that case-crossover studies employing the usual frequency comparison period overestimate risk compared to other recall time periods (‘yesterday’ or ‘last week’).

Materials and Methods

Literature Searches and Study Selection

A comprehensive search and review of peer-reviewed national and international literature was conducted on publications from 1970 to 2009. Using key word searches, electronic databases MEDLINE, Psychinfo and on-line journals were accessed to locate published material. Key terms, using Boolean operators were; (i) emergency room/emergency department/accident and emergency; and (ii) alcohol/drinking/alcohol drinking. The initial search string was: ‘emergency room’ OR ‘ER’ OR ‘emergency department’ OR ‘ED’ OR ‘injury’ AND ‘alcohol’. Since this initial search was too broad and yielded a very large number of results, we refined our search to focus on injury. In addition, we refined our initial search by limiting results to only peer reviewed journals, studies on humans and adults only, and studies published in English. For details on the flow of studies from literature search to final inclusion, please refer to Figure 1. A search of reference lists from recent relevant publications was also conducted to ensure all potentially suitable studies were identified. Internet search engines (Google and Google Scholar) and the National Drug Research Institute (NDRI) library were extensively searched to locate unpublished government reports and other information relevant to ED studies. Key national and international researchers were contacted to enquire of new or upcoming studies that could potentially be included. Early in the search process it became apparent that there were too few suitable ED-specific studies which investigated the relationship between recent alcohol use (i.e. as opposed to regular use) and non-injury conditions (e.g. hypertension, stroke) to conduct meta-analyses. This review therefore focused on injury, applying a broad, general definition without strict adherence to ICD codes or diagnostic criteria. This broad capture approach was chosen since many studies only use qualitative injury descriptions such as ‘falls’, ‘hits’ and ‘cuts’. Only ED studies that analysed self-reported alcohol use within the six-hour period prior to injury were included. For studies reporting results for males and females separately and in combination, we included results for combined estimates in an overall meta-analysis first; gender-separated results were then included in separate meta-analyses for males and females. For studies that reported odds ratios for all types of injury but by different injury categories and not for all injuries collectively, we reported the combined effect across injury categories. We excluded studies which reported results restricted to only one kind of injury e.g. sports or suicide. There were too few of these studies to group together for comparison; allowing them to be included in the meta-analysis among the more generalist studies would potentially bias results to reflect specific injuries rather than all injuries. We also excluded studies which drew cases from the general population rather than from ED populations. Studies were subject to the following specific inclusion criteria:

1. Injured samples must have been drawn specifically from ED populations, not the general population;

2. Controlled study design, i.e. either case-control or case-crossover;

3. Measured self-reported alcohol use within six hours of the injury (not within six hours of ED presentation);

4. Studies reporting only one type of injury, e.g. sports injuries or falls were excluded.

5. For studies with overlapping data (e.g., Cherpitel et al., 1988, 1993, 1997), we used the most recent results for a particular site using the largest pool of subjects, being careful to avoid duplication by excluding earlier sets of results.

Figure 1.

Flow of studies from literature search to inclusion in meta-analysis

Three coders underwent training during which they each coded four randomly selected studies from the included studies. Coding discrepancies were discussed until all issues were resolved and inclusion criteria were finalized. Using a standardised codebook, two coders independently coded the studies on: sample, study characteristics, demographics, nature of injury(s), alcohol consumption quantification, and matching and control variables. If a paper reported results separately for different countries, we treated each set of results as a different study. Similarly, it was possible to count several studies (‘sites’) from one paper if authors reported separate results for a case-control and a case-crossover study (Vinson et al., 2003). For one study that reported separate results for several injury categories, we calculated a combined estimate for all injuries (Kuendig et al., 2008). After all studies were coded, the two coders discussed all coding discrepancies in detail. The third trained coder checked the coding sheet for accuracy, completeness and consistency; any discrepancies were discussed until resolved. In consultation with study team and chief investigators, all coding discrepancies were resolved and the code book finalized. Since we only have data available on the upper age range for seven of the studies, non-age specific odd ratios were provided.

Analyses

Comprehensive Meta-Analysis version 2.0 was used. The summary measure was the odds ratio reported in studies. Studies were examined for heterogeneity and publication bias. For all analyses, a random effects (RE) model rather than a fixed effects (FE) model was chosen, since application of a FE models is based on the assumption that the studies being analysed are homogenous at the level of the study population effect sizes. This assumption could not be justified due to heterogeneity. Given different designs and settings, heterogeneity of effects across studies is common. In a RE analysis, the available studies are considered a sample of all possible studies on the association, with their estimates being subject to between-study variation (Hunter and Schmidt, 2000). However, a disadvantage of RE models is that they allow for between-study variation and partition the variance into within-study and between-study components. Hence, more weight is given to smaller studies than in the case of fixed effects modelling. We addressed this potential problem by careful assessment of publication bias. The RE model allows for between-study variability and incorporates this heterogeneity, however small, in the analysis of the overall strength of the effect. In addition, RE models were preferred since research indicates that FE models typically manifest substantial type I error bias in significance tests for mean effect sizes and for moderator variables. FE models, but not RE models, yield confidence intervals for mean effect sizes that are narrower than their nominal width, hence overstating the degree of precision in results (Hunter and Schmidt, 2000). Publication bias was assessed using funnel and precision plots and regression analysis (Sterne et al., 2000). Publication bias arises when trials with statistically significant results are more likely to be published and cited, and are preferentially published in English language journals and those indexed in database such as Medline; this could lead to publication of results that are systematically unrepresentative of the population of completed studies. The implications for meta-analyses are that combining only the identified published studies uncritically may lead to biased and over-optimistic conclusions (Sutton et al., 2000). That is, when research that is readily available differs in its results from the results of all the research that has been done in an area, reviewers of that research are in danger of drawing the wrong conclusion about what that body of research shows.

Meta-regression was conducted to estimate the impact of the moderator variables study design (e.g. case-control vs. case-crossover) and alcohol consumption recall period (usual frequency, ‘yesterday’ or ‘last week’) and to formally test whether there was evidence of different effects in these different subgroups of trials. The effect of gender was examined.

An overall meta-analysis without moderators was conducted first. This analysis compared only drinking any alcohol versus none in the six-hour period before the injury. The analysis was then stratified by study design, recall period and gender to examine sensitivity to these factors. Only few of the studies coded had reported age-adjusted models in their results; hence, age could not be used as a moderator variable.

Using study design type and recall period as moderator variables in mixed effects regressions, we also tested for significant differences in effect size magnitude when comparing studies by design (case-crossover vs. other) and by recall period (usual frequency vs. other methods). We particularly focused on the effects of not only study design per se, but also the impact of the recall period used in the studies; while separate results for case-control and case-crossover studies are informative, they are not showing the specific effect of the time window used for the recall of drinking prior to the injury.

Results

A total of 14 individual published research articles were included in the quantitative data synthesis after excluding twenty-one papers based on inclusion/exclusion criteria (Table 1 and Figure 1). All articles were published between 1988 and 2009 in English. Many of the study outcomes arose from research articles that presented results for more than one county and thereby more than one study outcome. The WHO (2007) study for instance presented individual outcomes for eight countries. Of the 14 research articles that met the inclusion criteria, 27 reported results were included in the meta-analysis. As discussed below, the results for India, arising from the WHO (2007) multi-nation study were later excluded given strong evidence of publication bias.

Table 1.

Description of studies meeting inclusion/exclusion criteria for overall meta-analysis

First Author	Year	Country	Study Design	N Cases	N Contr.	Weight	OR	95% CI Lower	95% CI Upper	P	Gender Contr.
Watt	2004	Australia	PCC	543	488	2.766	3.743	1.490	9.404	0.005	Yes
Vinson	2003	USA	PCC	2161	1856	4.465	3.099	2.300	4.200	0.000	No
Borges	1998	Mexico	PCC	756	756	3.786	6.753	3.901	11.691	0.000	Yes
Stockwell	2002	Australia	PCC	797	797	4.652	1.477	1.263	1.728	0.000	Yes
Kuendig	2008	Switzerland	EDCC	3592	3489	4.694	3.706	3.295	4.169	0.000	Yes
Cherpitel	1988	USA c	EDCC	1001	1378	4.569	1.363	1.099	1.691	0.005	No
Cherpitel	1988	USA d	EDCC	555	1278	4.569	3.004	2.422	3.727	0.000	No
Moskalewicz	2006	Poland	EDCC	485	270	4.053	1.768	0.996	3.122	0.013	No
Cherpitel	1999	Canada (AB)	EDCC	376	547	4.130	3.737	2.598	5.375	0.000	No
Cherpitel	1999	Canada (QC)	EDCC	349	436	4.040	1.270	0.853	1.894	0.239	No
Cherpitel	1997	USA e	EDCC	356	639	3.948	2.460	1.507	4.015	0.000	No
Cherpitel	1997	USA f	EDCC	1630	1416	4.465	0.677	0.515	0.891	0.005	No
Cherpitel	1997	USA g	EDCC	555	1095	4.341	2.160	1.548	3.014	0.000	No
Cherpitel	1993	Italy	EDCC	295	153	4.627	1.954	1.638	2.331	0.000	No
Vinson	2003	USA	CC Y	2517	2517	4.425	3.190	2.377	4.280	0.000	No
Vinson	1995	USA	CC Y	350	350	3.454	2.509	1.289	4.886	0.007	No
Gmel	2009	Switzerland	CC LW	486	486	3.840	3.004	1.770	5.100	0.000	No
Cherpitel	2005a	Poland a	CC UF	759	759	4.341	3.387	2.427	4.727	0.000	No
Cherpitel	2005b	Poland b	CC UF	759	759	4.465	5.207	3.957	6.851	0.000	No
WHO	2007	Argentina	CC UF	452	452	3.454	4.759	2.444	9.266	0.000	No
WHO	2007	Belarus	CC UF	510	510	2.576	24.78	9.120	67.328	0.000	No
WHO	2007	Brazil	CC UF	496	496	2.866	6.110	2.529	14.761	0.000	No
WHO	2007	Canada	CC UF	222	222	2.312	0.549	0.180	1.677	0.293	No
WHO	2007	Czech Rep.	CC UF	511	511	3.620	0.811	0.441	1.488	0.498	No
WHO	2007	Mexico	CC UF	456	456	3.288	6.821	3.303	14.086	0.000	No
WHO	2007	New Zealand	CC UF	160	160	2.440	7.389	2.564	21.293	0.000	No
WHO	2007	Sweden	CC UF	497	497	2.866	7.316	3.028	17.672	0.000	No
WHO	2007	India	CC UF	556	556	0.653	46.063	3.021	702.29	0.006	No

Note: ‘CC UF’ = case-crossover usual frequency; ‘CC Y’ = case-crossover yesterday; ‘CC LW’ = case-crossover last week; ‘EDCC’ = ED case-control; ‘PCC’ = population case-control

^a= Warsaw;

^b= Sosnowiec;

^c= Costa Contra;

^d= San Francisco;

^e= Jackson;

^f= Contra Costa;

^g= San Francisco

Based on self-reported drinking six hours prior to injury, the risk of injury at any level of drinking ranged from OR=2.44 for New Zealand to OR=46.06 for India. Only three studies reported ORs which were not statistically significant, two were from Canada (Cherpitel et al, 1999; WHO 2007) and one from the Czech Republic (WHO 2007).

Heterogeneity and publication bias

A significant Q-statistic (Q(26) = 356, p=0.000) indicated between-study heterogeneity. Hence, a random effects model was the preferred approach for meta-analyses. The funnel plot (not shown) indicated substantial asymmetry in the distribution of studies, indicating publication bias. This asymmetry was largely due to the WHO 2007 study results for India (OR=46.06). This study, while having a large effect size (log OR=3.829), also had the largest standard error (0.902). Removing this study produced a more normal funnel plot and the resultant improvement in the distribution of outcomes suggests that it would be beneficial to remove study the WHO result for India from further analysis, which was done. The funnel plot (not shown) also suggested that it might be beneficial to remove another study from the analyses – in this case, the WHO (2007) Belarus study. This study, while having a large effect size (log OR=3.210), also had a large standard error (0.510). However, due to the known high drinking levels in this country it was decided to retain Belarus in the analyses. After making these adjustments, further investigation of funnel plot asymmetry by use of precision (the inverse of the standard error) to predict the standardized effect showed that there was no longer a statistically significant publication bias. We applied Egger’s (1997) regression method to test for a linear association between the intervention effect and its standard error. In this test, the size of the treatment effect is captured by the slope of the regression line (B1) while bias is captured by the intercept (B0). After removing India, the intercept (B0) was 0.878, SE=1.321, 95% CI (−1.825, 3.581), with t=0.669, df=25. The two-tailed p-value was 0.509. This result suggests the absence of publication bias.

Assessment of Moderator Variables

The results of meta-regression by study design indicated that there was significant heterogeneity due to study design (i.e. ED case-controls, population case-controls and case crossover designs) and alcohol consumption recall period (e.g. usual frequency, ‘yesterday’ and ‘last week’).

Meta-analyses

Table 2 shows the pooled effect size for the overall meta-analysis including all 27 reported results (from 14 studies) prior to considering effects of moderator variables, and effect sizes when study design and recall period are considered. Testing for differences in the magnitude of effect sizes as a function of design features, we found significant differences in odds ratio magnitude when comparing studies by design, e.g. case-crossover vs. case-control (p=0.017) and by recall period in case crossover studies, e.g. usual frequency vs. studies using the ‘yesterday’ or ‘last week’ recall period (p=0.039). Gender was not a significant effect in the main analysis. We present separate estimates for females and males in Table 3. For the analysis by gender, five studies reporting separate estimates for females (Cherpitel, 1988; Cherpitel et al., 2005; Gmel et al., 2009; Stockwell et al., 2002; Watt et al., 2004) and six for males (Cherpitel, 1988; Cherpitel et al., 2005; Gmel et al., 2009; Moskalwicz et al., 2006; Stockwell et al., 2002; Watt et al., 2004) were available. Studies which provided gender-specific estimates found larger and significant effects for females, but small and non-significant effects for males.

Table 2.

Odds of injury arising in ED presentation within six hours of any alcohol consumption versus no alcohol consumption

	Effect Size			Test of Null
Model	N Studies (Results)	OR	Lower 95% CI	Upper 95% CI	Z	P
All studies	14 (27)	2.799	2.214	3.538	8.612	0.000
By study design
Case-crossover	5 (13)	3.815	2.646	5.499	7.175	0.000
ED case-control	5 (10)	1.977	1.385	2.821	3.754	0.000
Population case-control	4 (4)	3.145	1.583	6.247	3.272	0.001
By recall period reported
Usual Frequency	2 (10)	4.235	2.541	7.057	5.540	0.000
“Yesterday’ or ‘Last week’	12 (17)	2.320	1.789	3.008	6.349	0.000

Table 3.

Odds of injury arising in ED presentation within six hours of any alcohol consumption versus no alcohol consumption, by gender

	Effect Size			Test of Null
Groups	N Studies (Results)	OR	Lower 95% CI	Upper 95% CI	Z	P
Females	5 (5)	2.285	1.361	3.836	3.128	0.002
Males	6 (6)	1.071	0.715	1.605	0.334	0.737
Overall	11 (11)	2.242	1.618	3.106	4.853	0.000

Discussion

Our meta-analysis added to the landscape of extant alcohol and injury literature on several accounts. By only using patient’s self-reported alcohol consumption, rather than combining with a BAC measure, we have a more pure measure of alcohol impairment for the six hours prior to injury. That is, most studies use BAC measures that were taken too long after the injury, (e.g., >two hours after the injury). In these cases, BAC does not provide a very accurate measure of impairment. Further, we addressed an important methodological problem raised by Gmel (2010) and Gmel and Daeppen (2007), by focusing on the impact of study design on the magnitude of the effects in the alcohol and injury relationship. As Gmel (2010) and Gmel and Daeppen (2007) pointed out, particularly the use of case-crossover designs using the ‘usual frequency’ approach to measuring alcohol consumption may induce a downward bias in the effect size, since many respondents tend to under-estimate their alcohol consumption for this time period. While case-crossover designs using the ‘yesterday’ time window for comparisons may be methodologically stronger by reducing the recall bias in this shorter time span, they are still subject to recall bias (Gmel, 2010; Gmel and Daeppen, 2007). We investigated these potential sources of bias by comparing different recall periods as reported in the studies, not just study design, per se. Our findings of significant differences in effect size magnitude given study design and recall period clearly demonstrate the importance and practical impact of this issue, confirming that risk estimates are indeed higher for studies using the ‘usual frequency’ method. While our main meta-analysis did not find significant gender differences regarding the risk for injury, it is not necessarily the case that no such differences exist. Studies which provided gender specific estimates found larger and significant effects for women, but small and non-significant effects for males. In the overall analysis, no gender effect was found; this is an interesting finding given that women typically drink less than men and that the highest drinking categories in the studies are open-ended. This is suggestive of a substantial gender effects given the large difference in actual alcohol consumption. Further, studies which do report gender-specific results did find larger and significant effects for females. We also emphasize that the two studies that used population case-control design while controlling for context and other substance use (Stockwell et al., 2002; Watt et al., 2004) did find significant gender effects. These two studies do not suffer from the kinds of study design biases illustrated in the main analyses. These findings need to be replicated in future studies to arrive at conclusions regarding true gender differences in the alcohol and injury risk relationship. With a greater number of appropriately designed studies that control for social context and personality along the lines ofStockwell et al. (2002) andWatt et al. (2004), future meta-analyses of the risk relationship between alcohol and injury should detect gender differences.

Limitations

In our meta-analysis, one source of overall heterogeneity was that our set of studies contained both case-control and case crossover studies. Several differences in these studies can affect heterogeneity, the first of which is clearly the control group and the biases that result from this methodological difference. For example, in case-crossover designs, between subject confounding is well controlled; however, the issue of recall bias still remains (Borges et al., 2006). We addressed this issue not only by analysing these types of studies (case-control or case-crossover) separately, but by specifically analysing stratified by recall period. The finding of significant differences in effect size magnitude as a function of these methodological features confirms that the methods used to study the risk relationship between alcohol and injury can have a large influence on the size of the odds ratio and conclusions.

A further limitation involves statistical power. While our overall sample size was satisfactory, the number of studies within analyses by study type was limited. Therefore, combined estimates from these analyses need to be interpreted with caution. Similarly, our meta-analyses providing separate estimates from males and females need to be interpreted with the caveat in mind that we only had six studies providing separate results for females and seven studies with separate results for males. Case-control studies cannot control for context, only for individual personality factors such as gender and SES. However, injuries occur within a certain context, and in order to attempt controlling for such contextual factors, one should avoid the application of the ‘usual frequency’ method or longer recall periods such as ‘drinking last week’. A recommendation for future ED studies assessing the risk relationship between alcohol consumption and injury is that every study should control for contextual factors. Due to the limitation that only two of our studies included control for context and the other methodological issues regarding errors in direct comparisons of consumption by gender, the extent of true gender differences in this relationship is not yet clear. These two studies (Stockwell et al., 2002; Watt et al., 2004) reported marked gender differences in the risk relationship. A further limitation of the present meta-analysis is that we did not quantify alcohol consumption prior to injury other than the ‘any alcohol’ vs. ‘no alcohol’ dichotomy, and hence, do not present results for a dose-response relationship.

Many of the studies in our meta-analysis did not control for factors known to confound the relationship between alcohol and injury, such as usual drinking patters, drinking settings, other substance use and risk-taking behaviours (Cherpitel et al., 2012; Watt et al., 2004). However, our meta-analysis made several contributions to the alcohol and injury literature. The strengths of our meta-analysis were that we presented separate results for three commonly used study designs and by recall period, enabling a more informed comparison in terms of alcohol consumption and injury.

To conclude, our meta-analysis confirms clear biases in estimates of effect sizes for the risk relationship as a function of study design and alcohol consumption recall period chosen for comparative data on alcohol consumption unrelated to injury events. Our recommendation for future ED studies is to use shorter recall periods over the ‘usual frequency’ approach in order to obtain more reliable reports and more stable estimates. While ED case control studies tend to underestimate the magnitude of the effect size, case-crossover studies tend to overestimate. The usual frequency approach for recall tends to provide lower estimates of alcohol consumption. While costly to conduct, population case-control studies would be the approach of choice in order to circumvent the methodological challenges discussed in this study, since this approach provides the most reliable estimates of alcohol consumption, yielding more stable risk estimate.