What indeed can be tested with an instrumental variable?

An astronomer, a physicist and a mathematician are on a train in Scotland. The astronomer looks out of the window, sees a black sheep standing in a field, and remarks, ``How odd. All the sheep in Scotland are black!" ``No, no, no!" says the physicist. ``Only some Scottish sheep are black." The mathematician rolls his eyes at his companions' muddled thinking and says, ``In Scotland, there is at least one sheep, at least one side of which appears to be black from here some of the time." [1].

There is a fine line between useful technical advice and pedantry. Suppose that we perform an ideal two-armed randomized controlled trial with perfect blinding and perfect adherence, and show that participants randomly assigned to the treatment group have on average lower levels of blood pressure than those assigned to the control group. ``Wonderful!", says the epidemiology student. ``All participants that are given the treatment will have their blood pressure lowered!"

Of course, the student is wrong [2]. First, the randomized trial only assesses average causal effects. There is no guarantee that the treatment is effective for everyone in the population -- it may even raise blood pressure for some individuals. Secondly, the randomized trial was conducted in a particular sample of the population. Perhaps it was only conducted in middle-aged individuals -- so we have no evidence that the treatment is effective for older individuals. Going even deeper, perhaps the treatment was only effective in people with a particular characteristic -- maybe those individuals were common in the trial population, but are rare in the general population. Or maybe the treatment was effective at the time of the trial, but it will not be effective in the future -- perhaps it interacted with a particular environmental pollutant that was abundant at the time of the trial, but has since been banned. Of course, this is entirely speculation -- but the mathematician would be technically correct to say that all we can conclude from the randomized trial is that ``in the world, there are at least some people for whom this treatment appeared to be effective at lowering blood pressure at one point in time".

And, as Swanson, Labrecque and Hernán point out [3], in a Mendelian randomization investigation, one can conclude even less. In an ideal Mendelian randomization investigation with perfect instrumental variables, suppose that we observe an association between a genetic variant that is an instrumental variable for risk factor X, and blood pressure. We want to make the inference that intervening on X will lower blood pressure. However, there is a multitude of reasons why specific interventions on X may not lower blood pressure in clinical practice. As discussed by the authors, suppose that the genetic variant influences X, such that individuals in the genetically-defined subgroups of the population have different average levels of the risk factor X for their whole life course. Then it may be that intervening on X in childhood does lead to decreased levels of blood pressure, but once that the critical period of childhood has passed, then interventions on X do not affect blood pressure. A real example of this appears to be the effect of vitamin D on multiple sclerosis; Mendelian randomization suggests that vitamin D is a protective risk factor for multiple sclerosis [4], but studies of migrants suggest that sunlight exposure in childhood is the key predictive risk factor [5].

An important distinction here between Mendelian randomization and a randomized trial is that a randomized trial could in theory be designed to assess the causal effect of interest -- if we are planning to implement the intervention in a particular population in a particular way at a particular point in the life course, then we could design the randomized trial to be relevant for this intervention [6]. Even still, we would still have to make some assumptions that the result of the randomized trial generalizes to the clinical context in which the treatment is to be applied. But for Mendelian randomization, there will always be a disconnect between the genetic variant, which is determined at conception, and the treatment, which is usually to applied to a population of mature individuals [7]. There are also likely to be qualitative and quantitative differences between the way that the genetic variant influences the risk factor, and how the intervention influences the risk factor [8].

Epidemiologists are taught to consider the context of findings when assessing the relevance of evidence, and to synthesize (or `triangulate') sources of evidence that have different strengths and weaknesses [9]. In Mendelian randomization, in addition to categories of external validity relating to people and to situations [10], we also need to consider the nature of the intervention -- to what extent is the genetic variant a representative proxy for the particular intervention on the risk factor that is proposed? If it is a reasonable proxy (for example, the proposed intervention on the risk factor is to influence long-term usual levels of the risk factor in previously healthy individuals by a small amount [11]), then the Mendelian randomization investigation is likely to be informative. If it is not, then the Mendelian randomization investigation will be less informative about the impact of the proposed intervention. Of course, this discussion pre-supposes that there is a proposed intervention in the risk factor. If there is not, then even the target for inference -- the causal question of whether the risk factor influences the outcome or not -- is fuzzy and undefined [12].

The technical apparatus that the authors introduce to discuss the timing of genetic effects and interventions is a helpful codification to illustrate this intuition. It is certainly true that the observed null associations between genetic variants that influence CRP levels and CHD risk are more informative about the potential effect of long-term interventions on CRP levels (for example, by taking a polypill) than they are about interventions on acute levels of CRP. However, the question of timing is just part of a wider comparison that the analyst will have to make in judging to what extent the genetic variants are representative of any proposed intervention on the risk factor [13, 14]. Having said all this, we have to be careful not to throw out the baby with the bathwater -- although we cannot automatically jump from the presence of a genetic association to the effectiveness of a particular treatment, a positive finding from a Mendelian randomization investigation does make it more likely that intervening on the risk factor will have a causal effect on the outcome.

In short, even when we have a valid instrumental variable, Mendelian randomization is no panacea for causal inference. It can answer some, but not all causal questions. As Swanson, Labrecque and Hernán make clear, one reason for this limitation is timing -- particularly as most genetic variants ``switch on" at conception. In the vast majority of cases, there will be several other reasons. However, in this limitation there are also opportunities. If we can find genetic variants that have different trajectories of association with the risk factor over the life course, then we can exploit any differences in associations with the outcome to narrow down or identify critical windows for intervention. For example, we can compare the associations of BMI-related genetic variants that are associated with birthweight, versus those that are only associated with post-pubertal BMI. Alternatively, we can use these differences to identify potential mechanisms -- for example, by comparing genetic variants that influence BMI via increasing metabolic rate versus those that influence BMI via altering appetite [15]. While some genetic variants will be better or worse proxies for assessing the efficacy of particular interventions on a risk factor, we can exploit these differences to learn even more about causal effects, and to further sharpen our causal hypotheses.