Commentary: Building an evidence base for mendelian randomization studies: assessing the validity and strength of proposed genetic instrumental variables

The remarkable growth in the popularity of mendelian randomization (MR) studies—analyses using genetic polymorphisms as instrumental variables (IVs)—reflects both the importance of the causal inference problems addressed by MR and the potential power of the approach.¹ Although IV analysis is a well-established econometric tool, applications in health research using genetic information as IVs present novel challenges and opportunities. Papers such as that of Au Yeung et al., providing evidence on the characteristics of a proposed genetic IV for a specific phenotype-outcome association,² are important because they help establish a foundation of evidence relevant for many future MR studies. In this commentary, we discuss the value of integrating evidence from external sources, such as the Au Yeung et al. results in MR studies, with specific focus on using separate-sample IV (SSIV, sometimes called split-sample IV) in data from case-control studies.

Evaluating the validity of proposed genetic IVs entails first demonstrating that the genotype predicts the phenotype. A more serious challenge in evaluating an IV is to provide evidence for the assumption that the genotype has no causal links with the outcome of interest except via the phenotype of interest.^3,4 Although this cannot be conclusively proven, statistical, theoretical or biological evidence may bolster this assumption if such evidence suggests direct pathways are unlikely. External evidence such as that provided by Au Yeung et al. can be used to quantify the genotype-phenotype association; to support the assumption of no other causal pathways between the genotype and the outcome; and, via SSIV, to derive IV effect estimates.

IV analyses typically use two types of information: information on the association between the genotype and the phenotype, and information on the association between the genotype and the outcome. These two types of association can sometimes be evaluated using separate data sources. The SSIV method, described by Angrist et al. in the context of the two-stage least square (2SLS) IV estimator,⁵ is valid assuming that the associations observed in the data set used to estimate the genotype-phenotype association would also hold in the data set used to estimate the genotype-outcome association and vice versa.

The first stage of 2SLS IV estimator involves regressing the phenotype of interest (X) on the genetic IVs; for example, using three independent polymorphisms as IVs (Z₁-Z₃):

(1)

The predicted value of X is used as the independent variable in a second stage regression with the outcome (Y) as the dependent variable:

(2)

When external information about the association between proposed genetic instruments and the phenotype is available, the predicted value of X used in Equation 2 can be estimated based on this information. If the polymorphisms are thought to have independent, additive effects, the predicted value of X is simply the sum of the number of minor alleles times the estimated effect (as reported in the external study) of each minor allele on the phenotype. The coefficient for this weighted polygenic score predicting the outcome corresponds directly with the IV effect estimate. This approach has been implemented using beta weights from genome-wide association studies for body mass index,⁶ and, given the huge investments in estimating genotype-phenotype associations, it could be widely applicable.

SSIV can be used if the phenotype of interest was either not measured or was measured with substantial error in the data set in which the outcome was assessed. SSIV also has the advantage of remediating weak-instruments bias, so there is less concern about using multiple polymorphisms as IVs, even if many of these polymorphisms are only weakly associated with the phenotype. To see this, note that if a set of genetic polymorphisms has no causal effect on a phenotype, a regression model using these polymorphisms to predict the phenotype will nonetheless typically be weakly predictive within any finite sample. The prediction will improve as more non-causal polymorphisms are used in the regression model, just as adding any random number as a predictor in a regression model always improves prediction R² within the sample. The predicted phenotype value from such a model will be correlated with unmeasured causes of the phenotype within that sample, including those confounding factors that influence both the phenotype and the outcome. When this predicted phenotype value is used in the second stage of the 2SLS, it will also predict the outcome, due to its correlation with the confounders. As a result, the standard 2SLS IV estimate is biased towards the confounded conventional effect estimate when there are multiple IVs that have very small or no effects on the phenotype. This phenomenon is eliminated when the two stages are estimated in separate samples, because the association is not structural: the linear combination of the genotypes that by chance best predicts the phenotype in one sample is unrelated to the confounders of the phenotype-outcome association in the second sample.

In health research, data are often drawn from case-control studies rather than representative samples and SSIV may be especially valuable in this context. We compare below some possible approaches to deriving IV estimates in case-control data when (i) the outcome is rare in the population and (ii) the phenotype of interest is continuous.

If case-control status defines the outcome in an MR analysis, standard 2SLS estimation as described above will generally not consistently estimate the phenotype-outcome causal effect. A notable exception occurs under the null hypothesis of no causal phenotype-outcome effect, in which case 2SLS rejects the null at the nominal type 1 error rate. An alternative IV analysis potentially applicable when the binary outcome is rare in the target population is to replace the second stage with a logistic regression.⁷ If data are drawn from a random sample, the resulting estimator is consistent for the multiplicative causal association between the phenotype and the outcome, provided: (i) the causal effect is constant across levels of unmeasured confounders, i.e. there is no effect modification by unmeasured confounders, and (ii) the residuals for the first stage regression are homoscedastic^4,8 Palmer et al. compare several IV estimators appropriate for the situation of a random sample with a continuous phenotype and binary outcome, noting the target parameter (marginal or conditional causal odds ratio or risk ratio) and the assumptions under which each estimator is consistent for the target parameter.⁹

Under case-control sampling, however, this approach is not necessarily correct; although if the first stage regression is performed in controls only (and therefore gives results similar to the first stage regression in the population), the approach is approximately correct.¹⁰ This strategy has the disadvantage of losing information because cases are not used in the first stage estimates. Some information may be recovered in the first stage by pooling cases and controls and including case-control status as a covariate in the linear regression model,¹¹ but the approach is sensitive to the rare disease assumption.¹² To implement this approach, the fitted values fed into the second stage must be computed setting the case-control variable to its reference value (i.e. as if each observation were a control) for all subjects in the sample. In the event of heterogeneity in the IV-phenotype association by case-control disease status, the approach can be generalized by including an interaction between the IV and disease status in the first stage regression. As before, the fitted values fed into the second stage should be computed for all subjects upon setting their disease status to the reference level.

Alternatively, if the outcome is not rare in the population, nested case-control data can be analysed by inverse-probability weighting both stages of 2SLS by the probability of selection into the sample.¹⁰ Selection probabilities are typically known by design or can be estimated in nested case-control studies. For example, if all cases in a large cohort are selected into the nested case-control study, along with 10% of non-cases in the cohort, the 2SLS could be implemented by assigning each case a weight of 1 and each non-case a weight of 10. However, this approach can be inefficient compared with the conditional approach described above, although some efficiency may be recovered by augmenting inverse probability estimating equations.¹³

SSIV provides a third alternative for implementing MR in case-control data sets. The first stage estimates of the association between the genotype and phenotype are obtained from other data sources and used to construct the expected value of the phenotype for every observation in the case-control data set. A logistic regression estimated with the expected phenotype value predicting the outcome then estimates the causal odds ratio (approximating the causal risk ratio) assuming the outcome is rare and the genotype-phenotype association in the external source matches the association in the target population from which the case-control data were drawn. Additional IV techniques for binary outcomes and case-control studies, including structural mean models and generalized method of moments, can be found elsewhere.^9,10

Simple modifications of the approaches above are applicable in secondary analyses of case-control data, when the outcome of interest is not the outcome used to define ‘case’ status. The two-stage approach applying logistic regression in the second stage may be used by including an indicator for case-status in both analysis stages. However, as before, predicted values of the phenotype estimated from the first stage regression must again be evaluated with case-control status set to its reference value. This two-stage approach recovers the phenotype-outcome multiplicative association provided (i) neither case-control status (i.e. cases as used to select the data) nor unmeasured confounders interact with the phenotype on a multiplicative scale, and (ii) case-control status does not interact with the IV in the first stage regression (on a linear scale). The SSIV, using external information to estimate the first stage regression, is also appropriate when the outcome of interest is not the condition originally used to define cases, but again the second stage must be estimated including an indicator for case-status.

In all two-stage analyses, the same covariates must be included in both stages. This presents a difficulty in SSIV models because the same covariates may not be available in both data sets. In studies that aim to provide evidence on the validity of a proposed genetic IV, it is therefore valuable to present evidence on the genotype-phenotype association with several alternative covariate sets. Finally, as noted above, many approaches rely on ‘no-interaction’ assumptions, so evidence on the plausibility of such assumptions, i.e. whether the genotype-phenotype associations are consistent across a range of characteristics, will be valuable.

Randomized controlled trials will remain the sine qua non of causal inference in health studies, but MR and related approaches offer valuable techniques for assessing causal hypotheses. Establishing a base of evidence regarding proposed genetic IVs, as in the Au Yeung et al. article, is a critical step forward to facilitate evaluation of IV validity and IV effect estimation in other studies.

Funding

The authors gratefully acknowledge financial support from the National Institute on Mental Health (MH092707-01) and the National Institute of Environmental Health Sciences (1R21ES019712-01) at the National Institutes of Health.

Conflict of interest: None declared.