Commentary: Reporting and assessing evidence for interaction: why, when and how?

Thompson¹ noted in 1991 that, although more than a decade had passed since the first discussion of ‘interaction’ in the epidemiological literature, debate had by then subsided and few clear conclusions had emerged. He concluded:

Unfortunately, choice among theories of pathogenesis is enhanced hardly at all by the epidemiological assessment of interaction … What few causal systems can be rejected on the basis of observed results would provide decidedly limited etiological insight.

This conclusion probably represented the consensus of opinion at that time. Yet 20 years later, discussion and reports of interactions pervade the literature. In this journal alone, two papers giving guidelines and recommendations for reporting and assessing interactions have appeared within weeks of one another. Knol and VanderWeele² considered only the reporting of interactions. Boffetta et al.³ primarily discuss the assessment of evidence, but, since they are much concerned with the ‘cumulative’ evidence following meta-analysis or systematic review, they inevitably touch on the reporting of interactions.

One could point to several developments that have contributed to this reawakening of interest. The first of these has arisen from recent work by the statisticians in ‘causal modelling’, which has led to new, deterministic, definitions of mechanistic interaction. One such approach derives from the earlier work of Rothman⁴ on component causes, whereas the other is based on the currently popular focus on counterfactual outcomes. In the former approach, interaction between two causal factors is defined as the presence of a sufficient cause, which involves both factors as component causes, whereas from the latter viewpoint, interaction implies the existence of people in the population who would not have developed disease unless they had been exposed to both factors. These viewpoints have been shown to be mutually consistent^5–7 and, subject to certain assumptions, are also consistent with statistical definition of interaction as deviation from additivity of effects on risk. Broadly, the same conclusions follow from stochastic models for independent causes, originally discussed by Rothman⁸ and Miettinen⁹ and recently revisited from the standpoint of directed acyclical graphical models by VanderWeele and Robins.¹⁰

In common with many discussions of interaction and mechanism, Boffetta et al.³ and Knol and VanderWeele² ignored the role of time (particularly age). For chronic degenerative diseases, this must be considered in any analysis, but it is not straightforward. Greenland and Poole⁵ briefly discussed this problem in the context of stochastic sufficient cause models, pointing out that most commonly used additive models impose rigid constraints on the form of cause-specific hazard functions. Such constraints are avoided in the non-parametric additive hazards model of Aalen¹¹ (extended to case–control data by Borgan and Langholz¹²). However, the time-dependent aspect of the problem seems to have been ignored in deterministic causal theories.

Although recent work has established a more rigorous basis for the idea of mechanistic interaction, its identification in many circumstances with non-additivity for effects on risk had already been recognized for many years before Thompson came to his pessimistic conclusion. We can only assume that he regarded establishment of the existence of such interaction as providing ‘decidedly limited etiological insight’. So, what has changed to re-ignite epidemiologists’ enthusiasm for interaction? The answer must surely be the greater availability of genetic data. Thus, discussion of gene–gene interaction often identified with the earlier (and better defined) concept of epistasis pervades the recent genetic epidemiology literature, and few epidemiological grant applications now fail to identify the establishment of ‘gene–environment interaction’ as a primary aim. Yet much of this discussion is as careless in its use of terms as the early epidemiological literature that first prompted debate about the topic 40 years ago; an interaction trumpeted in the title of a paper more often than not turns out to represent a significant interaction term in a logistic regression model, despite the fact that this, in general, has few implications for mechanism. Statistical interaction between two genes in the logistic regression model is often described as epistasis, although the multiplicative model for joint effects of two genes, which is approximately equivalent to a ‘main effects’ logistic model, has previously been used as model for epistasis.^13,14 Against this background, perhaps there is a place for the publication of some guidelines.

Perhaps, the most important service that could be provided by the issuing of guidelines would be to foster greater clarity in presentation of precisely ‘why’ the ‘interaction’ reported is of interest. As is argued by Knol and VanderWeele² this, in turn, has implications for ‘how’ it should be reported. In this respect, Boffetta et al.³ contribute little, after the now familiar agonizing about how difficult it is to define ‘interaction’, finishing with a conclusion remarkably similar to Thompson's of 20 years before:

It is not always clear whether or not sensible biological conclusions can come from a particular statistical formula of interaction.

The discussion continues as if this warning had never been issued. Thus, there is detailed discussion of bias in relation to the interaction parameter in the multiplicative model, and of designs, such as the case-only design in which ‘only’ this parameter can be estimated, despite the admission that interpretation of this parameter is problematic (inference concerning sufficient cause interaction in the latter case has been discussed by VanderWeele et al.¹⁵). One must question the rationale for the proposal of prescriptive guidelines for reporting and assessing a phenomenon whose scientific meaning is unclear. If a paper is to report an interaction as a major finding, then it must make clear precisely what sort of interaction is reported and what can be legitimately concluded. We now have a rigorous theory which allows us, at least in some circumstances, to test a well-defined concept of no mechanistic interaction in the sufficient cause framework. But the fact remains that this will often be of ‘decidedly limited interest’. Although it would be rash to claim that the hypothesis of truly independent sufficient causes will never be of interest, one suspects that it will only rarely be plausible. Indeed, it would usually be more interesting if one could show that two causes did indeed operate through independent mechanisms. But, of course, absence of statistically significant interaction does not allow such a conclusion, any more than its presence, on whatever scale, implies that the factors operate on the same biological pathway.

It is often argued that, regardless of any direct causal interpretation, interaction on the additive scale is important from a public health point of view. Even this is over-stated; as pointed out by Clayton and McKeigue,¹⁶ the argument for targeted intervention does not only depend on non-additivity of effects—the ‘prevention paradox’¹⁷ remains under multiplicative models for accrual of risk.

Despite these reservations, presence or absence of statistical interaction is an important part of any data analysis as it is related to the goodness of fit of the model adopted for risk (either explicitly or implicitly) and, therefore, to the accurate assessment of risks for different risk factor profiles. This brings us to the question of ‘when’ statistical interactions should be reported. The usual criterion for this is Occam's razor, applied either by a classical significance test for interaction or by some optimal prediction criterion such as the Akaike Information Criterion.¹⁸ As has been widely recognized, interaction parameters are estimated precisely much less than the main effects so that they are only rarely significant in a single study. However, Boffetta et al.³ are concerned with systematic review, which requires that studies are reported in such a way that evidence can be combined across studies. This raises the possibility that interaction could be insufficiently convincing to report in an individual study, but statistically significant when evidence is combined over studies. Boffetta et al.³ recommend publication of web supplementary tables of data on gene–environment joint effects and, since this recommendation comes at the end of a discussion of the dangers of selective reporting, we can only assume that they are proposing that Occam's razor should not apply in this context. But, clearly reporting of all non-significant interactions in the expectation of later meta-analysis is not feasible. Although not explicitly suggested, it is implied that data on interactions should be presented regardless of their significance when there is sufficient prior expectation that interaction is present. However, in the absence of clear relationship between plausible mechanisms and statistical interaction in a given risk model, how should such prior expectations be informed? Ultimately, there would seem to be no solution to this problem; meta-analysis with an emphasis on joint effects of two or more factors will, of necessity, require assembly of the relevant raw data.

The problem of how to report significant interactions remains, and was discussed in some detail by Knol and VanderWeele.² This essentially concerns the choice of parametrization of an interaction term in a (generalized) linear model. In the case of two binary risk factors, Knol and VanderWeele distinguish between an ‘interaction’ parametrization, which considers the effect of each risk factor combination with one combination taken as baseline, and an ‘effect modification’ parametrization, which reports the effect of one factor for each level of the other. Both of these parametrizations include one or both of the main effects, and address slightly different questions. It is arguable whether it is necessary to publish guidelines that suggest how authors should lay out their tables to best make their point but, nevertheless, this article contains some useful insights. However, it is worth noting that the ‘interaction’ parametrization results in parameter estimates, which are strongly intercorrelated as a result of the use of a shared baseline. This can be misleading and seriously limits subsequent use of results in meta-analyses. In this context, perhaps the idea of ‘floating absolute risk’¹⁹deserves mention. Although this has been criticized,²⁰ its limitations are now better understood and presentation of the ‘quasi-variances’²¹ will often be helpful.

Although reporting and assessment of evidence for interaction are not the same thing, they are clearly linked. Opinions will, no doubt, differ as to whether or not the publication of guidelines to aid either process is necessary or even useful. In addressing reporting, Knol and VanderWeele² have at least clearly defined what they mean by ‘interaction’ and have not made unrealistic claims for its interpretation. In contrast, the guidelines for assessment of evidence suggested by Boffetta et al.³ are in danger of adding to the continuing confusion.

Funding

Wellcome Trust Principal Research Fellowship (to D.C.); Juvenile Diabetes Research Foundation (to D.C.). The Cambridge Institute for Medical Research is in receipt of a Wellcome Trust Strategic Award (079895).

Conflict of interest: None declared.