Recommendations for presenting analyses of effect modification and interaction

Abstract

Authors often do not give sufficient information to draw conclusions about the size and statistical significance of interaction on the additive and multiplicative scales. To improve this, we provide four steps, template tables and examples. We distinguish two cases: when the causal effect of intervening on one exposure, across strata of another factor, is of interest (‘effect modification’); and when the causal effect of intervening on two exposures is of interest (‘interaction’).

Assume we study whether X modifies the effect of A on D, where A, X and D are dichotomous. We propose presenting: (i) relative risks (RRs), odds ratios (ORs) or risk differences (RDs) for each (A, X) stratum with a single reference category taken as the stratum with the lowest risk of D; (ii) RRs, ORs or RDs for A within strata of X; (iii) interaction measures on additive and multiplicative scales; (iv) the A–D confounders adjusted for.

Assume we study the interaction between A and B on D, where A, B and D are dichotomous. Steps (i) and (iii) are similar to presenting effect modification. (ii) Present RRs, ORs or RDs for A within strata of B and for B within strata of A. (iv) List the A–D and B–D confounders adjusted for.

These four pieces of information will provide a reader the information needed to assess effect modification or interaction. The presentation can be further enriched when exposures have multiple categories. Our proposal hopefully encourages researchers to present effect modification and interaction analyses in as informative a manner as possible.

Background

Effect modification or interaction is often studied in epidemiological research. A survey of 225 cohort and case–control studies showed that 61% of the studies addressed effect modification or interaction in their publications.¹ However, the vast majority of these studies did not give sufficient information to the reader to draw conclusions on the size and statistical significance of the interaction on an additive and multiplicative scale. Only 11% of the studies presented individual effects of both exposures and the joint effect of both exposures; most studies used multiple reference categories which makes it impossible to assess overall interaction measures.

For over 30 years, there has been general consensus in the epidemiological community that measuring interaction on the additive scale is most appropriate for assessing the public health importance of interactions.^2–4 The relative excess risk due to interaction (RERI) is often considered the standard measure for interaction on the additive scale with case–control studies,⁵ although some advocate for the synergy index of additivity.⁶ Nevertheless, among a random sample of 50 case–control and 25 cohort studies in the five most highly ranked epidemiological journals, only one reported a measure of the synergy index and not one among the sample reported RERI.¹ Even when the study design allows for the estimation of risk differences, estimates are often reported using odds ratios (ORs) because logistic regression is used for covariate adjustment and then interaction is often not reported on the additive scale. There is clearly need for more comprehensive guidelines for reporting interaction analyses.

Earlier reporting proposals were made by Botto and Khoury for gene–environment interactions,⁷ and in the STROBE (STrengthening the Reporting of OBservational studies in Epidemiology) recommendations.⁸ STROBE recommendations state that interaction analyses should be preferably presented as separate effects of the two risk factors and their joint effect using one reference category, because this gives enough information to the reader to calculate interaction on an additive and multiplicative scale.

However, the earlier proposals did not distinguish between effect modification and interaction.⁹ Earlier proposals also did not suggest giving confidence intervals (CIs) for interaction on the additive and multiplicative scale. Here, we propose a format for presenting interaction analyses that is easy to interpret and will allow readers to assess interaction measures of potential interest. We encourage reporting measures and CIs of both additive and multiplicative interaction.

We distinguish cases in which (i) the causal effect of one exposure within strata of another exposure is of interest, referred to as ‘effect modification’ and (ii) the causal effects of two exposures together are of interest, referred to as ‘interaction’. For a more elaborate discussion on the distinction between effect modification and interaction and its consequences for analysis, we refer to VanderWeele⁹ and VanderWeele and Knol.¹⁰ Essentially, for effect modification there is only one hypothetical intervention under consideration, whereas for interaction there are two; for effect modification there is only one set of confounding factors to consider, those for the primary exposure of interest; for interaction, there are two sets to consider, those for both of the exposures. To illustrate this, consider a trial on the effect of supportive housing for homeless adults with chronic illness.¹¹ Suppose that the effect of supportive housing on the number of hospital days were greater in adults with employment, but that this was because employment status was correlated with mental health, and that it was mental health (rather than employment status) that had a causal effect on hospital days. It would still be the case that the effect of the housing intervention varied by employment status (i.e. ‘effect modification’ would be present) and the effectiveness of treatment could be increased by targeting employed persons. However, it might not be the case that if we intervened on employment status this would increase the effectiveness of the housing programme (i.e. ‘causal interaction’ may be absent). This is because employment may be confounded by mental health. If interest is in identifying target groups (e.g. whether supportive housing should be targeted at adults with employment), one is studying effect modification, and control for confounding of the effect modifier or second exposure is not necessary. If interest is in intervening on both exposures, (e.g. whether the effect of the housing programme would increase if persons were given employment along with the housing programme), one is studying interaction, and control for confounding of the second exposure is necessary.

Effect modification

Suppose we study whether the effect of A (the exposure of interest) on D (the outcome) is modified by X (the potential effect modifier), where A, X and D are dichotomous. We propose the following four steps in presenting the results of this analysis which will allow a reader to obtain the information needed to assess effect modification (Table 1):

1. Present relative risks (RRs), ORs or risk differences (RDs) with CIs for each stratum of A and X with a single reference category (possibly taken as the stratum with the lowest risk of D).

2. Present RRs, ORs or RDs with CIs for A within strata of X.

3. Present measures of effect modification on both additive (e.g. RERI^12,13) and multiplicative scales with CIs and P-values.

4. List the confounders for which the relation between A and D was adjusted. If A has more than two levels, additional columns could be added (see the Appendix available as Supplementary Data at IJE online for further discussion).

Although not essential for the interpretation of effect modification results, authors are encouraged to report the number of subjects with and without the outcome in each cell as well and use the 4 by 2 table lay-out as recommended in STROBE.⁸ This will add additional transparency for the reader, as it allows direct interpretation of the raw data, comparisons with other data and recalculations. If the study design allows calculation of absolute risks, these can preferably be presented in each of the cells of the table along with or instead of RRs or ORs.

Table 1.

Template table to presenting results of analyses on effect modification where A is the exposure of interest and X is the potential effect modifier

	A = 0		A = 1
	N with/without outcome	RR/OR/RD (95% CI); P	N with/without outcome	RR/OR/RD (95% CI); P	RR/OR/RD (95% CI); P for A = 1 vs A = 0 within strata of X
X = 0		Reference
X = 1

Measure of effect modification on additive scale (95% CI); P.

Measure of effect modification on multiplicative scale (95% CI); P.

RR/OR/RD is adjusted for …

When possible, we recommend that RRs be reported rather than ORs. The OR is a ‘non-collapsible’ measure so that marginal and conditional measures may not coincide—the measure of effect modification may thus vary depending on the covariates for which control has been made. When the outcome is rare, ORs and RRs will nearly coincide. However, when the outcome is common, the two may diverge and using ORs may exaggerate interaction measures. In a case–control study with incidence density sampling, the OR can be interpreted as a rate ratio, circumventing this problem.¹⁴

Example

A cohort study by Knol et al.¹⁵ investigated whether the risk of antidepressant use (the exposure of interest A) on diabetes (the outcome D) was modified by the chronic disease score (the potential effect modifier X). The chronic disease score is a measure of the chronic disease status among drug users and can be considered as an indicator of an individual's morbidity and overall health status. The score was dichotomized as 0 and ≥1, where 0 means that no chronic disease is present. Following the four steps described above the results would be presented as in Table 2:

1. RRs with CIs and P-values are presented for antidepressant use only (1.16; 0.95–1.42), for chronic disease score of ≥1 only (1.55; 1.30–1.84), and for antidepressant use and chronic disease score of ≥1 (2.07; 1.73–2.47), where no antidepressant use and chronic disease score of 0 is the reference category, as this stratum gives the lowest risk of diabetes.

2. RRs with CIs and P-values are presented for antidepressant use (the exposure of interest) in strata of chronic disease score (the potential effect modifier). These RRs show that the risk of antidepressant use on diabetes is 1.16 in subjects with a chronic disease score of 0 and 1.34 in subjects with a chronic disease score of ≥1.

3. The RERI was calculated as 2.07 − 1.55 − 1.16 + 1 = 0.36 with a 95% CI obtained by the delta method¹² of −0.003 to 0.73. This means that there were strong indications that the estimated effect on the additive scale of antidepressant use with a chronic disease score of ≥1 was larger than the estimated effect of antidepressant use with a chronic disease score of 0, so there is positive effect modification of antidepressant use across strata of the chronic disease score on an additive scale. The measure of interaction on a multiplicative scale, the ratio of RRs in strata of the chronic disease score, was 1.15 (95% CI 0.89–1.49). This means that there were some indications that the estimated joint effect of antidepressant use on the risk ratio scale with a chronic disease score of ≥1 was larger than the estimated effect of antidepressant use with a chronic disease score of 0.

4. The relation between antidepressant use and diabetes was adjusted for age, sex and benzodiazepine use. These are taken to be the confounding factors for antidepressant use; if only effect modification (the effect of antidepressant use as it varies across strata of chronic disease score) is of interest, then it is not necessary to consider confounding factors for chronic disease score.

Table 2.

Example—modification of the effect of antidepressant use on diabetes by chronic disease score

	No antidepressant use		Antidepressant use
	N with/without diabetes	RR (95% CI)	N with/without diabetes	RR (95% CI)	RRs (95% CI) for antidepressant use within strata of chronic disease score
Chronic disease score of 0	243/24 195	1.0	153/15 470	1.16 (0.95–1.42); P = 0.150	1.16 (0.95–1.42); P = 0.150
Chronic disease score of 1 or more	338/11 260	1.55 (1.30–1.84); P < 0.001	246/8611	2.07 (1.73–2.47); P < 0.001	1.34 (1.13–1.58); P = 0.001

Measure of effect modification on additive scale: RERI (95% CI) = 0.36 (−0.003–0.73); P = 0.052.

Measure of effect modification on multiplicative scale: ratio of RRs (95% CI) = 1.15 (0.89–1.49); P = 0.282.

RRs are adjusted for age, sex and benzodiazepine use.

Interaction

Suppose now instead we study the interaction between A and B (the two exposures of interest) on D (the outcome), where A, B and D are dichotomous. We propose the following four steps in presenting the results of this analysis which will allow a reader to obtain the information needed to assess interaction (Table 3):

1. Present RRs, ORs or RDs with CIs and P-values for each stratum of A and B with a single reference category (possibly taken as the stratum with the lowest risk of D).

2. Present RRs, ORs or RDs with CIs and P-values of the effect of A on D in strata of B and of B on D in strata of A.

3. Present measures of effect modification on both additive (e.g. RERI^12,13) and multiplicative scales with CIs and P-values.

4. List the confounders for which the relation between A and D and for which the relation between B and D were adjusted.

Note that Steps 1 and 3 are exactly the same for studying interaction or effect modification. Step 2 for studying interaction, however, requires presenting the effects in strata of B and strata of A, whereas Step 2 for studying effect modification only includes presenting the effects in strata of X (the effect modifier). Step 4 for studying interaction requires that confounders for both the relation between A and D and between B and D are reported, whereas Step 4 for studying effect modification only includes listing the confounders for the relation between A and D.

Table 3.

Template table to presenting results of analyses on interaction where A and B are the two exposures of interest

	A = 0		A = 1
	N with/without outcome	RR/OR/RD (95% CI); P	N with/without outcome	RR/OR/RD (95% CI); P	RR/OR/RD (95% CI); P for A = 1 vs A = 0 within strata of B
B = 0		Reference
B = 1
RR/OR/RD (95% CI); P for B = 1 vs B = 0 within strata of A

Measure of interaction on additive scale (95% CI); P.

Measure of interaction on multiplicative scale (95% CI); P.

RR/OR/RD is adjusted for …

Although not essential for the interpretation of interaction results, authors are encouraged to report the number of subjects with and without the outcome in each cell as well, and use the 4 by 2 table lay-out as recommended in STROBE.⁸ This will add additional transparency for the reader, as it allows direct interpretation of the raw data, comparisons with other data about either exposure, and recalculations. If the study design allows calculation of absolute risks, these should preferably be presented in each of the cells of the table. As before, we recommend that RRs rather than ORs be presented when possible.

Example

Van Gils et al.¹⁶ investigated the interaction between dietary intake of vitamin E (exposure of interest A) and a polymorphism in a gene coding for proteins in the DNA repair system (XRCC1 Codon 399 genotype) (exposure of interest B) on the risk of prostate cancer (outcome D) in a case–control study. Following the four steps described above the results would be presented as in Table 4:

1. ORs with CIs are presented for the Arg/Arg genotype only (1.04; 0.42–2.60), for low dietary vitamin E intake only (1.22; 0.53–2.82) and for the Arg/Arg genotype and low dietary vitamin E intake (2.40; 1.02–5.63), where Arg/Gln + Gln/Gln genotype and high dietary vitamin E intake is the reference category.

2. ORs with CIs and P-values are presented for the relation between Arg/Arg genotype and prostate cancer in strata of dietary vitamin E intake, and ORs with CIs and P-values are presented for the relation between low dietary vitamin E intake and prostate cancer in strata of the genotype.

3. The RERI was 1.14 (95% CI: −0.57 to 2.85), meaning that there were some indications that the estimated joint effect on the additive scale of vitamin E and the Arg/Arg genotype together was greater than the sum of the estimated effects of vitamin E alone and the Arg/Arg genotype alone so that there was positive interaction on the additive scale. The measure of interaction on a multiplicative scale, the ratio of ORs, was 1.89 (95% CI 0.56–6.34), meaning that there were some indications that the estimated joint effect on the OR scale of vitamin E and the Arg/Arg genotype together was greater than the product of the estimated effects of vitamin E alone and the Arg/Arg genotype alone so that there was positive interaction on the multiplicative scale.

4. The relations between Arg/Arg genotype and prostate cancer and between low dietary vitamin E intake and prostate cancer were adjusted for age, ethnicity, first-degree relative with prostate cancer, education, ever been a farmer, BMI, total energy intake, total fat intake and intake of other antioxidants (vitamin C, vitamin A, β-carotene and lycopene). If causal interaction were of interest these factors would have to suffice to control for confounding for the effect of vitamin E and for the effect of XRCC1 Codon 399 genotype.

Note that the authors did not control for population stratification which could confound the effect of genotype. If polymorphisms for XRCC1 Codon 399 are regarded as the true causal variants and no genetic confounding is present then the analyses could be considered as giving measures of causal interaction. Otherwise, if there were genetic confounding, one might then instead want to present these results as effect modification with the effect of vitamin E varying across strata defined by genotype. More generally, in gene–environment interaction analyses, one has to be clear whether the genetic variant can be interpreted as causal or not, i.e. whether there is confounding or not. Without controlling for confounding factors for genetic variants, a joint analysis of both factors can always be interpreted as effect modification of the environmental factors across strata of the genetic factor so long as control is made for the confounders of the environmental exposure. If control is also made for genetic confounding so that associations between the genetic variant and disease reflect the true effects of the genetic factor then the analysis can be interpreted as one of causal interaction.

Table 4.

Example—interaction between XRCC1 codon 399 genotype and dietary vitamin E intake on the risk of prostate cancer

	XRCC1 codon 399 genotype
	Arg/Gln + Gln/Gln		Arg/Arg
	N cases/controls	OR (95% CI)	N cases/controls	OR (95% CI)	OR (95% CI) for Arg/Arg within strata of vitamin E
High vitamin E intake	17/46	1.0	14/44	1.04 (0.42–2.60); P = 0.93	1.04 (0.42–2.60); P = 0.93
Low vitamin E intake	22/57	1.22 (0.53–2.82); P = 0.65	24/33	2.40 (1.02–5.63); P = 0.04	1.97 (0.89–4.41); P = 0.10
ORs (95% CI) for vitamin E within strata of genotype		1.22 (0.53–2.82); P = 0.65		2.30 (0.93–5.74); P = 0.07

Measure of interaction on additive scale: RERI (95% CI) = 1.14 (−0.57 to 2.85); P = 0.19.

Measure of interaction on multiplicative scale: ratio of ORs (95% CI) = 1.89 (0.56 to 6.34); P = 0.30.

ORs are adjusted for age, ethnicity, first-degree relative with prostate cancer, education, ever been a farmer, BMI, total energy intake, total fat intake and intake of other antioxidants.

Conclusion

The way of presenting results that we propose gives all relevant data to interpret effect modification and interaction analyses. Editors of epidemiology journals could encourage authors to present their results in this format. The STROBE recommendations propose to present interaction analyses as separate effects of the two risk factors and their joint effect with one reference category, because this gives sufficient information to the reader to recalculate interaction on whatever scale is preferred.⁸ Our proposal goes one step further, as it also recommends presentation of the effect estimates of one factor across strata of another and presentation of measures of and CIs for interaction on additive and multiplicative scales.

A recent survey of cohort and case–control studies indicated that measures and P-values or CIs for interaction on the additive or multiplicative scale were often not reported.¹ Measures and CIs on the additive scale were especially likely to be omitted, even in publications in the most prominent epidemiological journals. This is particularly worrisome given the biological and public health importance of the additive scale. For public health purposes, the additive scale indicates whether the effect of a risk factor would be greater in one subpopulation than in another and is thus useful in targeting specific populations and in resource allocation. A measure of interaction on the additive scale, such as RERI, can be used to assess whether there is synergism between the two exposures as defined in the sufficient cause model (i.e. individuals for whom the outcome would occur if both exposures are present but not if only one or the other were present).^17–19 If both exposures have neutral or causative effects for all individuals (i.e. the effects are monotonic), RERI > 0 implies such synergism;^18,20 without monotonicity, one can still test for synergism by testing RERI > 1.^19,20 That measures of interaction on an additive scale are not reported in most analyses of effect modification and interaction deprives the reader of important information.

Perhaps part of the reason that measures and CIs of interaction on the additive scale are often not reported is that these are not immediately given in the output of logistic regression procedures with standard statistical software. Nevertheless, there are publicly available SAS programs that will compute measures and CIs for interaction on the additive scale²¹ as well as Excel spreadsheets that can be used to automatically do these computations from standard output given by SAS, Stata or SPSS.²² We provide another easy-to-use spreadsheet tool in the Appendix available as Supplementary Data at IJE online (available also on the first author's website) that can be used in conjunction with the output of any standard statistical software to calculate measures for additive interaction, and corresponding CIs and P-values using the delta method.¹² This same tool could also be used in conjunction with a Cox proportional hazards model to produce measures of additive interaction,^23,24 and to calculate additive interaction between continuous exposures.²⁵ These tools can be easily employed by epidemiologists and ought to be routinely included in epidemiological methods courses that discuss interaction. Other methods than the delta method for calculating CIs for measures of additive interaction are also available.^26–29 We strongly encourage reporting both additive and multiplicative interaction estimates and CIs whenever interaction or effect modification is of interest.

We have focused on the RERI in this paper as a measure of additive interaction. The synergy index is also sometimes considered⁶ (which is the ratio of the extent to which the risk of the doubly exposed group exceeds 1 to the extent to which the sum of the singly exposed groups exceeds 1). However, this measure should only be used if the doubly unexposed group is the one with the lowest absolute risk.³⁰ Although some prefer the synergy index, arguments for using RERI are given above and elsewhere.^31,32 Both could potentially be reported.

Our proposal will hopefully encourage researchers to present interaction analyses in as informative a manner as possible.

Supplementary Data

Supplementary Data are available at IJE online.

Funding

This study was performed within the context of the Escher project (T6-202), a project of the Dutch Top Institute Pharma. National Institutes of Health (grant R01 ES017876 to T.V.).