A Bayesian approach to Mendelian randomization using summary statistics in the univariable and multivariable settings with correlated pleiotropy

Summary

Mendelian randomization uses genetic variants as instrumental variables to make causal inferences on the effect of an exposure on an outcome. Due to the recent abundance of high-powered genome-wide association studies, many putative causal exposures of interest have large numbers of independent genetic variants with which they associate, each representing a potential instrument for use in a Mendelian randomization analysis. Such polygenic analyses increase the power of the study design to detect causal effects; however, they also increase the potential for bias due to instrument invalidity. Recent attention has been given to dealing with bias caused by correlated pleiotropy, which results from violation of the “instrument strength independent of direct effect” assumption. Although methods have been proposed that can account for this bias, a number of restrictive conditions remain in many commonly used techniques. In this paper, we propose a Bayesian framework for Mendelian randomization that provides valid causal inference under very general settings. We propose the methods MR-Horse and MVMR-Horse, which can be performed without access to individual-level data, using only summary statistics of the type commonly published by genome-wide association studies, and can account for both correlated and uncorrelated pleiotropy. In simulation studies, we show that the approach retains type I error rates below nominal levels even in high-pleiotropy scenarios. We demonstrate the proposed approaches in applied examples in both univariable and multivariable settings, some with very weak instruments.

We propose a Bayesian framework for performing Mendelian randomization with potentially pleiotropic genetic instruments. Our proposed method, MR-Horse, is shown to produce unbiased causal effect estimates while avoiding inflated false positive rates, even with high levels of pleiotropy and weak instruments. We further extend the approach to the multivariable setting.

Introduction

Mendelian randomization is a technique that exploits genetic variation to make causal inferences on the effect of one or more exposures on an outcome of interest.¹ The underlying idea is that, because of the Mendelian laws of inheritance, a genetic variant that is associated with an exposure of interest can serve as a proxy for the exposure. Such a proxy should be less susceptible to the confounding and reverse causation that can inhibit causal inference from observational data. Methods for Mendelian randomization have been developed that can combine information from multiple genetic variants associated with the same exposure. Furthermore, many of these methods can be performed using only summary statistics taken from genome-wide association studies (GWASs), removing the need to access sensitive individual-level data. Under the two-sample framework, summary statistics for the exposure and outcome can be taken from separate, ideally non-overlapping, samples.² Thus, any combination of exposure and outcome for which there exists publicly available GWAS summary statistics can potentially be considered in a Mendelian randomization study. However, in order to provide robust evidence for causal effects, strong assumptions must be made pertaining to the relationships between the genetic variants and the exposures, outcome, and any confounder variables.

Mendelian randomization can be considered as a form of instrumental variables analysis, where each genetic variant must satisfy the three instrumental variables assumptions. In the single exposure case, these assumptions are that each genetic variant:

• (1)is associated with the exposure,
• (2)is not associated with the outcome via any confounding pathway, and
• (3)does not affect the outcome other than via the exposure.

The relationships between a valid genetic variant and other relevant traits is illustrated graphically in the directed acyclic graph (DAG) in Figure 1A.³

Figure 1.

Directed acyclic graphs

Shown is the relationship between the jth genetic variant ($G_j$), exposure (X), outcome (Y), and unmeasured confounders (U) when: (A) $G_j$ is a valid instrument, (B) $G_j$ is invalid due to a direct effect on Y, (C) $G_j$ is invalid due to both a direct effect on Y and an indirect effect via U, (D) $G_j$ is invalid due to both a direct effect on Y and being caused by U, and (E) $G_j$ is invalid due to either (i) a direct effect on Y or (ii) both a direct effect on Y and an indirect effect via U but with no direct effect on X.

A common source of instrument invalidity is pleiotropy, which is where genetic variants associate with multiple traits. If a genetic instrument associates with a predictor of the outcome via a causal pathway that bypasses the exposure, then this will open up a pleiotropic pathway and invalidate the third instrumental variables assumption. Another source of invalidity is population structure, in which there are differences in the genetic distributions among subgroups of the population being studied. This results in confounding of the genetic variant-outcome relationship and invalidates the second instrumental variables assumption. Such confounding may occur if there is population stratification, dynastic effects, or assortative mating.⁴

Much attention has been given in the literature to developing methods for Mendelian randomization that are robust to instrument invalidity. Many of these methods have been developed motivated by providing robustness against pleiotropy. Often, the InSIDE (instrument strength independent of direct effect) assumption is required for these methods to give valid results, which means that any pleiotropic effects are independent of the genetic variant-exposure associations.⁵ Methods that require InSIDE will not provide robustness to population structure, since the confounding pathway may create a correlation between the genetic effects on the exposure and the direct effects on the outcome. Recently, more attention has been given to methods that can also handle violations of the InSIDE assumption, a scenario referred to as correlated pleiotropy.⁶ However, methods that provide such robustness typically suffer from a combination of inflated type I error rates, low power, or further restrictive assumptions.

In this paper we propose a Bayesian approach to Mendelian randomization that is robust to both uncorrelated and correlated pleiotropy. We show that this approach is able to provide unbiased and efficient causal effect estimation without inflated type I error rates in very general settings, with favorable performance in comparison with existing methods. A further advantage of the proposed method is that it is easily generalized to the multivariable Mendelian randomization paradigm, which has far fewer robust methods than the single exposure case. We demonstrate the approach in an applied example that looks at the relationship between combinations of four exposures (LDL cholesterol, triglycerides, fasting glucose, and birth weight) and three outcomes (coronary artery disease, type 2 diabetes, and asthma). We further consider an example looking at the effects of intelligence, years of education, and household income on the risk of Alzheimer disease in a multivariable model.

Material and methods

Models for instrument invalidity

We consider a potentially causal exposure X for an outcome Y with unmeasured confounders U. We denote by θ the causal effect of X on Y and assume that this effect is linear and homogeneous in the population (that is, that there is no effect modification). This latter assumption allows for causal effect estimation in the instrumental variables framework.

The relationships between a valid genetic instrument, $G_j$, and the exposure, outcome, and unmeasured confounders are shown in Figure 1A. If we have a single such instrument, we can estimate the causal effect using the Wald ratio, ${\hat{\beta }}_{Yj}/{\hat{\beta }}_{Xj}$, where ${\hat{\beta }}_{Xj}$ and ${\hat{\beta }}_{Yj}$ are estimates of the associations between $G_j$ and X and Y, respectively. These association estimates are typically obtained by using linear regression for continuous traits and logistic regression for binary traits. If we have J valid instruments, the inverse-variance weighted (IVW) method fits the model

$${\hat{\beta }}_{Yj}\sim N\left(\theta {\hat{\beta }}_{Xj},{\sigma }_{Yj}^2\right),j=1,\dots ,J\text{,}$$ (Equation 1)

where ${\sigma }_{Yj}$ is the standard error of the jth genetic variant-outcome association estimate.⁷ Note that the standard error of the jth genetic variant-exposure association estimate, ${\sigma }_{Xj}$, does not appear in the IVW model, which reflects the fact that this approach implicitly assumes that the genetic associations with the exposure are measured without error.

Most pleiotropy robust methods for Mendelian randomization make the assumption that some of the instruments are valid and allow for varying levels of invalidity. MR-PRESSO assumes that a small number of instruments are invalid, which will result in outliers in the Wald ratio estimates.⁸ Under the InSIDE assumption, MR-PRESSO tests for outliers and estimates the causal effect using IVW with the identified outliers removed. If at least half of the instruments are valid, a weighted median of the Wald ratios gives a consistent estimate of the causal effect, whether or not InSIDE is satisfied.⁹

Figure 1B illustrates the case where there is a pleiotropic pathway to the outcome that does not pass via X or U (although it may go via other unmeasured traits that do not associate with the exposure). In this case, the direct effect ${\tilde{\alpha }}_j$ is independent of the instrument strength ${\tilde{\beta }}_{Xj}$, and hence InSIDE is met. The causal effect can be estimated using the MR-Egger method, which fits the model

$${\hat{\beta }}_{Yj}\sim N\left(\theta {\hat{\beta }}_{Xj}+{\alpha }_0,{\sigma }_{Yj}^2\right),j=1,\dots ,J\text{.}$$ (Equation 2)

The intercept term ${\alpha }_0$ represents the average pleiotropic effect across all variants. If this average effect is equal to zero, then it is said that pleiotropy is balanced. If the average effect is non-zero, the pleiotropy is said to be directional. Under the MR-Egger model, potentially all variants may be invalid, but only in such a way that the InSIDE assumption is satisfied.

Figure 1C shows the case where the genetic variant has a pathway to the outcome via one or both of a direct path and a path that passes through a confounder of the exposure-outcome relationship. This latter pathway creates correlated pleiotropy, or, in other words, violates InSIDE. One commonly used model for this scenario is

$${\hat{\beta }}_{Yj}\sim N\left(\theta {\hat{\beta }}_{Xj}+{\alpha }_j,{\sigma }_{Yj}^2\right),j=1,\dots ,J\text{,}$$ (Equation 3)

where ${\alpha }_j$ represents the effect ${\tilde{\alpha }}_j+{\gamma }_j{\delta }_Y$ and ${\hat{\beta }}_{Xj}$ is an estimate of ${\beta }_{Xj}={\tilde{\beta }}_{Xj}+{\gamma }_j{\delta }_X$. Under the assumption that some of the instruments are valid and some are invalid, Kang et al.¹⁰ and Rees et al.¹¹ proposed estimating θ using penalized regression with a lasso-style penalty on the ${\alpha }_j$ terms. In this way, the estimates of ${\alpha }_j$ relating to the valid instruments are shrunk toward zero. A post-lasso approach fits the IVW model using only the instruments whose ${\alpha }_j$ estimates were shrunk to zero in the initial lasso regression. Although these lasso-based approaches provide unbiased effect estimates, they can suffer from substantially inflated type I error rates due to the post-selection inference problem. Kang et al.¹⁰ showed that the causal effect is identifiable in the model given in Equation 3 when a plurality of instruments are valid. That is, if there are groups of variants for which the single-instrument causal effect estimate is the same, the largest of these groups is the set of valid instruments. Alternative methods that make the plurality valid assumption include the Contamination Mixture method (ConMix)¹² and MR-Mix,¹³ which are based on fitting mixture distributions to the Wald ratio estimates, and weighted mode approaches.^14,15

Xue et al.¹⁶ proposed a method that fits the model

$${\hat{\beta }}_{Yj}\sim N\left(\theta {\beta }_{Xj}+{\alpha }_j,{\sigma }_{Yj}^2\right),{\hat{\beta }}_{Xj}\sim N\left({\beta }_{Xj},{\sigma }_{Xj}^2\right),j=1,\dots ,J\text{,}$$ (Equation 4)

using constrained maximum likelihood, referred to as MR-cML. This method provides consistent causal effect estimation under the plurality valid assumption. Similar to the lasso regression approaches, MR-cML maximizes the likelihood in such a way that the ${\alpha }_j$s pertaining to valid instruments will tend toward zero. However, through the use of model averaging, it suffers less from inflated type I errors. Furthermore, the model accounts for the uncertainty in the ${\beta }_{Xj}$ estimates. The MR-RAPS method is also based on the model given in Equation 4, where pleiotropy is accounted for using a robust loss function. However, MR-RAPS makes the InSIDE assumption, requires balanced pleiotropy, and was shown to have lower power than MR-cML by Xue et al.¹⁶

Bayesian methods have been far less utilized in the Mendelian randomization literature, despite a number of potential advantages over frequentist approaches in fitting a model such as that given in Equation 4. Berzuini et al.¹⁷ developed a Bayesian approach where parameters representing pleiotropic effects are given a shrinkage prior. This has the effect of shrinking these terms pertaining to the valid instruments toward zero, similar to the lasso-based approaches. The use of credible intervals obtained from the posterior distribution was shown to provide better coverage than the weighted median method, suggesting that the Bayesian approach is better able to estimate the uncertainty in the effect estimates. The method requires individual-level data and an assumption akin to InSIDE. Zhao et al.¹⁸ (BWMR) and Bucur et al.¹⁹ (BAYESMR) have proposed Bayesian approaches using summary-level data. Similar to Berzuini et al.,¹⁷ these approaches place a shrinkage prior on the ${\alpha }_j$ terms and make the InSIDE assumption.

Recently developed approaches aimed at the correlated pleiotropy case include CAUSE,⁶ MR-Corr2,²⁰ and MR-CUE,²¹ which each utilize Bayesian methods. MR-Corr2 uses a spike-and-slab prior for pleiotropic terms, similar to BAYESMR, and also includes a parameter to account for correlated pleiotropy. However, it does not allow for both uncorrelated and correlated pleiotropy among the variants. CAUSE and MR-CUE make the assumption, illustrated in Figure 1E, that genetic instruments are either directly associated with the exposure or only associated with the exposure via a confounder. Xue et al.¹⁶ showed that CAUSE is particularly sensitive to violations of this assumption. Another feature of these approaches is that they utilize genome-wide summary statistics, as opposed to just summary statistics relating to a curated set of instruments. The genome-wide summary statistics are used to estimate nuisance parameters that are inputs to the Bayesian models.

Figure 1D shows the case where correlated pleiotropy is caused by a confounder of the genetic variant-outcome relationship, for example, due to population structure. Although the cause of instrument invalidity differs from that shown in Figure 1C, we can model it in the same way. We thus consider Equation 4 to be a very general parametric model for Mendelian randomization with pleiotropy. It can represent invalidity due to both pleiotropy and population structure, and it does not make the assumption of CAUSE and MR-CUE that the instruments cannot both be directly associated with the exposure and subject to correlated pleiotropy.

Univariable Mendelian randomization

Suppose we have, for J genetic variants, summary statistics of their association with the exposure and outcome. We denote the estimate of the association between the jth variant and the exposure and outcome by ${\hat{\beta }}_{Xj}$ and ${\hat{\beta }}_{Yj}$, respectively, and the standard errors of the estimates by ${\sigma }_{Xj}$ and ${\sigma }_{Yj}$. We model the association estimates by

$${\hat{\beta }}_{Yj}\sim N\left(\theta {\beta }_{Xj}+{\alpha }_j,{\sigma }_{Yj}^2\right)$$

$${\hat{\beta }}_{Xj}\sim N\left({\beta }_{Xj},{\sigma }_{Xj}^2\right)\text{.}$$

We place a jointly normal prior on $\left({\beta }_{Xj},{\alpha }_j\right)$,

$$\left(\begin{array}{c}{\beta }_{Xj}\\ {\alpha }_j\end{array}\right)\sim N\left(\left(\begin{array}{c}\mu \\ 0\end{array}\right),\left(\begin{array}{cc}{s}_X^2& {\rho }_j{s}_X{\phi }_j\tau \\ {\rho }_j{s}_X{\phi }_j\tau & {\phi }_j^2{\tau }^2\end{array}\right)\right)\text{,}$$

and we use the following weakly informative priors:

$$\begin{array}{l}\mu \sim N\left(\mathrm{0,1}\right)\\ s_X^2\sim N^{+}\left(\mathrm{0,1}\right)\\ \theta \sim N\left(\mathrm{0,1}\right)\text{,}\end{array}$$

where $N^{+}\left(·,·\right)$ is the half-normal distribution (that is, the truncated normal distribution bounded below at 0).

In this model, the ${\beta }_{Xj}$ term represents the effect of the jth genetic variant on the exposure, and the ${\alpha }_j$ term represents the effect of the variant on the outcome that bypasses the exposure. If the jth variant is a valid instrument, then the true value of ${\alpha }_j$ is zero. Otherwise, the variant is pleiotropic, and the pleiotropic effect is accounted for by a non-zero ${\alpha }_j$. If the jth variant is either a cause of, or caused by, U, then ${\beta }_{Xj}$ and ${\alpha }_j$ will be correlated. This correlation is modeled by the ${\rho }_j$ parameter. Thus, the model can account for valid instruments (${\alpha }_j=0$), uncorrelated pleiotropy (${\alpha }_j\ne 0,{\rho }_j=0$), and correlated pleiotropy (${\alpha }_j\ne 0,\rho \ne 0$). Any prior distribution for ${\rho }_j$ that restricts its values to the interval $\left(-\mathrm{1,1}\right)$ may be used and should be informed by domain knowledge of the likely biological mechanisms underlying the genetic variant associations. For our simulation study and applied examples, we set ${\rho }_j=2r_j-1$ where $r_j\sim \text{Beta}\left(\mathrm{10,10}\right)$. This places the majority of the mass in the prior distribution around zero, reflecting the fact that many of the genetic variants will either be valid instruments or have pleiotropic effects that are close to uncorrelated with their exposure association. However, this prior still allows for substantial correlation when this is present.

For the final two parameters in the model, we use the priors

$$\begin{array}{l}{\phi }_j\sim {\text{Cauchy}}^{+}\left(\mathrm{0,1}\right)\\ \tau \sim {\text{Cauchy}}^{+}\left(\mathrm{0,1}\right)\text{,}\end{array}$$

where ${\text{Cauchy}}^{+}\left(·,·\right)$ is the half-Cauchy distribution. Under this parametrization, the marginal distribution of ${\alpha }_j$ is the horseshoe prior.²² The horseshoe is a shrinkage prior that will shrink some of the ${\alpha }_j$s toward zero, while non-zero ${\alpha }_j$s will escape shrinkage better than when using a Laplace prior (which is equivalent to lasso regression). Thus, the horseshoe prior is less likely to underestimate the value of non-zero ${\alpha }_j$s. The τ parameter controls the global level of shrinkage such that a lower value will shrink more of the ${\alpha }_j$s close to zero. In our case, we assign τ a half-Cauchy prior and allow its posterior distribution to be estimated from data. However, if there was prior knowledge on the level of sparsity, this could be incorporated by adjusting this prior or setting a fixed value.²³ The horseshoe prior has previously been proposed for pleiotropic effects in Mendelian randomization by Berzuini et al.¹⁷ for the individual-level data setting.

In practice, we have implemented the method using BUGS via the rjags package in R.²⁴ We take $\mathrm{10,000}$ samples from the posterior distribution (with a burn-in of $\mathrm{10,000}$ iterations) from 3 chains, which, in all cases reported in the simulation study and applied examples, resulted in R-hat values for the parameters of interest close to 1, indicating good convergence. The causal effect estimate is the mean of the posterior samples, and the 95% credible interval is the lower and upper 2.5th percentiles. We call our method MR-Horse (after the horseshoe prior).

Multivariable Mendelian randomization

Multivariable Mendelian randomization considers the effect of multiple exposures on the outcome.^25,26 Originally motivated as a method for accounting for measured pleiotropy, it can also be used to simultaneously estimate causal effects of multiple correlated exposures,²⁷ to perform risk factor selection,²⁸ and in mediation analysis.²⁹ A genetic variant is a valid instrumental variable for multivariable Mendelian randomization if it is associated with at least one exposure, it is not associated with the outcome via any confounding pathway, and it does not affect the outcome other than via one or more of the measured exposures.

The multivariable IVW approach fits the model given in Equation 1, but where the ${\hat{\beta }}_{Xj}$s are vectors of the association estimates between the jth genetic variant and each exposure. There are far fewer pleiotropy robust methods for multivariable Mendelian randomization compared with the univariable setting. Commonly used approaches include multivariable versions of MR-Egger³⁰ and MR-PRESSO, a multivariable median-based approach,³¹ and GRAPPLE,³² akin to a multivariable version of MR-RAPS. Recently, a multivariable version of MR-cML has been developed.³³

Our proposed model for univariable Mendelian randomization can be easily generalized to the multivariable case as follows. We denote by ${\beta }_{Xj·}$ the vector of associations between the jth genetic variant and the K exposures, and by ${\hat{\beta }}_{Xj·}$ their estimates that have covariance matrix ${\mathrm{\Sigma }}_{Xj}$. As with the standard errors in the univariable case, the ${\mathrm{\Sigma }}_{Xj}$s are assumed known. In practice, they can be estimated either using individual-level data or from GWAS summary statistics (see, for example, Sanderson et al.³⁴ and Grant et al.³⁵). We fit the model

$${\hat{\beta }}_{Yj}\sim N\left({\theta }^{\prime }{\beta }_{Xj·}+{\alpha }_j,{\sigma }_{Yj}^2\right)$$

$${\hat{\beta }}_{Xj·}\sim N\left({\beta }_{Xj·},{\mathrm{\Sigma }}_{Xj}\right)\text{,}$$

where the kth element of the vector θ represents the direct causal effect of the kth exposure on the outcome. We place a joint prior on the $\left({\beta }_{Xj·},{\alpha }_j\right)$ as follows:

$$\left(\begin{array}{c}{\beta }_{Xj·}\\ {\alpha }_j\end{array}\right)\sim N\left(\left(\begin{array}{c}\mu \\ 0\end{array}\right),\left(\begin{array}{cc}{V}_X& \text{diag}{\left(V_X\right)}^{1/2}{\rho }_j{\phi }_j\tau \\ {\left\{\text{diag}{\left(V_X\right)}^{1/2}\right\}}^{\prime }{\rho }_j{\phi }_j\tau & {\phi }_j^2{\tau }^2\end{array}\right)\right)\text{,}$$

and we use the following priors:

$$\mu \sim N\left(0,I_K\right)$$

$$V_X\sim N^{+}\left(0,I_K\right)$$

$${\theta }_k\sim N\left(\mathrm{0,1}\right)$$

$${\phi }_j\sim {\text{Cauchy}}^{+}\left(\mathrm{0,1}\right)$$

$$\tau \sim {\text{Cauchy}}^{+}\left(\mathrm{0,1}\right)\text{.}$$

As in the univariable case, this parametrization puts a horseshoe shrinkage prior on the ${\alpha }_j$s with global shrinkage controlled by the τ parameter. The correlation between the genetic variant-exposure effects and the pleiotropic effects are accounted for via the ${\rho }_j$ parameters, where, again, we set ${\rho }_j=2r_j-1$ with $r_j\sim \text{Beta}\left(\mathrm{10,10}\right)$.

As before, we have implemented the method using BUGS and the rjags package in R, with $\mathrm{10,000}$ samples taken from the posterior distribution (with a burn-in of $\mathrm{10,000}$ iterations) from 3 chains. In all cases reported in the simulation study, R-hat values for the parameters of interest were close to 1, indicating good convergence. We call the multivariable method MVMR-Horse.

Simulation study

Primary simulation parametrization

In order to examine the performance of our method in comparison with alternatives, we generated data based on the simulation parametrization of Xue et al.,¹⁶ with some parameters adapted to ensure levels of instrument strength and variation of the exposure explained that is typical of realistic settings. The data generating mechanism is

$$U=\sum _{j=1}^J{\gamma }_j{G}_j+{\epsilon }_U$$

$$X=\sum _{j=1}^J{\tilde{\beta }}_{Xj}{G}_j+U+{\epsilon }_X$$

$$Y=\theta X+\sum _{j=1}^J{\tilde{\alpha }}_j+U+{\epsilon }_Y\text{,}$$

where $G_j\sim \text{Binomial}\left(2,f_j\right)$, $f_j\sim \text{Uniform}\left(\mathrm{0.1,0.3}\right)$, ${\epsilon }_U,{\epsilon }_Y\sim N\left(\mathrm{0,1}\right)$, and ${\epsilon }_X\sim N\left(0,{\xi }^2\right)$, independently. For each j, ${\tilde{\beta }}_{Xj}$ was generated uniformly on the interval $\left(-0.15,-0.05\right)\cup \left(\mathrm{0.05,0.15}\right)$, ${\gamma }_j\sim \text{Uniform}\left(-\mathrm{0.1,0.1}\right)$, and ${\delta }_X={\delta }_Y=1$. We considered scenarios where either 20%, 40%, or 60% of variants were invalid instruments. Unlike Xue et al.,¹⁶ we considered both balanced and directional pleiotropy scenarios. In the balanced pleiotropy case, ${\alpha }_j\sim \text{Uniform}\left(-\mathrm{0.2,0.2}\right)$, and in the directional pleiotropy case, ${\alpha }_j\sim \text{Uniform}\left(-\mathrm{0.1,0.3}\right)$, when the jth variant was invalid, and in both cases ${\alpha }_j=0$ when the jth variant was valid. The causal effect was either $\theta =0.1$ or 0. Note that under this parametrization, InSIDE was violated in all scenarios. In each of 1,000 replications, two independent samples of sizes $n_X$ and $n_Y$ were generated. The genetic association estimates with the exposure were taken in the sample of size $n_X$ and with the outcome in that of size $n_Y$. The association estimates and their standard errors were obtained using simple linear regression of the exposure and outcome on each genetic variant in turn.

We considered three primary settings. In the first, we set $J=100$, ${\xi }^2=1$, and $n_X=n_Y=\mathrm{50,000}$. This generated strong genetic instruments (with mean F-statistic across all scenarios between 77.4 and 85.0), which explained a high proportion of the variation in the exposure (with mean $R^2$ value across all scenarios between 15.4% and 16.9%). In the second setting, we set $J=100$, ${\xi }^2=4$, $n_X=\mathrm{5,000}$, and $n_Y=\mathrm{50,000}$. This generated weak genetic instruments (mean F-statistic between 4.4 and 4.7) but that still explained a reasonably high proportion of the variance in the exposure (mean $R^2$ between 8.6% and 9.3%). In the third setting, we set $J=20$, ${\xi }^2=1$, and $n_X=n_Y=\mathrm{50,000}$. This generated strong genetic instruments (mean F-statistic between 87.7 and 98.2) that explained a reasonably low proportion of the variance in the exposure (mean $R^2$ between 3.5% and 3.9%).

Comparison methods

We compared our approach with standard IVW estimation and also an oracle estimator that performed IVW using only the truly valid instruments. We used a weighted median method, MR-Median, which requires half of the variants to be valid but does not make the InSIDE assumption nor requires balanced pleiotropy. We used BWMR to compare with another Bayesian approach, noting that this method assumes InSIDE. Finally, we compared with MR-cML as well as an extension to this approach, MR-cML-DP. These latter two methods both make the plurality valid assumption and aim to shrink estimates of the pleiotropic effects pertaining to valid instruments toward zero. The extension uses data perturbation (DP) to produce multiple perturbed samples (the default, which we use, being 200 samples) and then takes averages across the samples for estimation of the causal effect and standard error. This significantly increases the computational time but has been shown to reduce bias and type I error rates.

Other common methods not considered include MR-Egger, MR-RAPS, and MR-PRESSO (InSIDE and balanced pleiotropy) and also ConMix, MR-Mix, MR-Mode, and MR-Lasso (plurality valid, can handle some directional pleiotropy). In the simulation study of Xue et al.,¹⁶ MR-cML generally outperformed each of these methods in terms of bias, power, and type I error rates. Because our simulation study is largely based on that of Xue et al.,¹⁶ it is expected that MR-cML would be the best performing out of these, and thus for brevity we don’t compare with all these methods in our study. Note that MR-cML also performed favorably compared with MR-Median in the study of Xue et al.;¹⁶ however, we retain MR-Median here so we have another comparison method that does not require balanced pleiotropy or InSIDE.

Genetic variants acting on the exposure via a single mechanism

Under the primary simulation parametrization, each genetic variant acts directly on the exposure, and each invalid genetic variant also acts on the exposure indirectly via the unmeasured exposure-outcome confounders. To explore a more parsimonious setting where genetic variants act on the exposure through only one of these mechanisms, we repeated the simulation study in the first setting (that is, with $J=100$, ${\xi }^2=1$, and $n_X=n_Y=\mathrm{50,000}$), but where either ${\beta }_{Xj}=0$ or ${\gamma }_j=0$, with an equal probability of both, for all j relating to invalid instruments. In addition to the methods used in the primary simulation study, we also performed MR-CUE, using its default settings, in this scenario, which explicitly makes this assumption.

Genome-wide simulation study

We performed a simulation study that simulated a set of summary statistics representing genetic associations from independent variants across the genome, in order to compare our approach with CAUSE. We simulated $J=\mathrm{100,000}$ genetic variant associations such that either $p=100$ or 20 variants were expected to have non-zero association with the outcome, setting the heritabilities of X and Y to $h_X=h_Y=0.3$, respectively. We considered scenarios where either 20%, 40%, or 60% of these variants were expected to be pleiotropic. To do this, we first simulated ${\overline{\beta }}_{Xj}$, ${\overline{\alpha }}_j$, and ${\overline{\gamma }}_j$ from the joint mixture distribution

$$\left(\begin{array}{c}{\overline{\beta }}_{Xj}\\ {\overline{\alpha }}_j\\ {\overline{\gamma }}_j\end{array}\right)\sim \sum _{i=1}^3{\pi }_{i}N\left(\left(\begin{array}{c}0\\ 0\\ 0\end{array}\right),\left(\begin{array}{ccc}{c}_{Xi}& 0& 0\\ 0& c_{\alpha i}& 0\\ 0& 0& c_{\gamma i}\end{array}\right)\right)\text{,}$$

where

$$\left(c_{X1},c_{X2},c_{X3}\right)=\left(\frac{h_X}{p},\frac{2h_X}{3p},0\right)\text{,}$$

$$\left(c_{\alpha 1},c_{\alpha 2},c_{\alpha 3}\right)=\left(0,\frac{h_Y-{\theta }^2{h}_X}{p},0\right)\text{,}$$

$$\left(c_{\gamma 1},c_{\gamma 2},c_{\gamma 3}\right)=\left(0,\frac{h_X}{3p},0\right),\text{and}$$

$$\left({\pi }_1,{\pi }_2,{\pi }_3\right)=\left(\frac{p\left(1-q\right)}{J},\frac{pq}{J},1-\frac{p}{J}\right)\text{,}$$

and q is the proportion of variants associated with the exposure that are invalid. These were converted to non-standardized effect sizes by ${\beta }_{Xj}=\left({\overline{\beta }}_{Xj}+{\overline{\gamma }}_j\right)/\sqrt{2f_j\left(1-f_j\right)}$ and ${\alpha }_j=\left({\overline{\alpha }}_j+{\overline{\gamma }}_j\right)/\sqrt{2f_j\left(1-f_j\right)}$. The genetic association estimates were then generated by ${\hat{\beta }}_{Xj}\sim N\left({\beta }_{Xj},s^2\right)$, ${\hat{\beta }}_{Yj}\sim N\left(\theta {\beta }_{Xj}+{\alpha }_j,s_j^2\right)$, where $s_j^2=1/\left(2n{f}_j\left(1-f_j\right)\right)$, $n=\mathrm{50,000}$. We considered scenarios where $\theta =0.1$ or 0.

For each variant, we computed a p value using ${\hat{\beta }}_{Xj}$ and $s_j$. We performed the same methods used in the primary simulation, taking as inputs variants with p value $<5\times {10}^{-8}$ (that is, which pass the conventional genome-wide significance threshold). We also performed CAUSE using all simulated genetic variants, and with its default threshold p value $<0.001$ as classifying that a variant is associated with the exposure. Due to the extra computational time required to run the simulations, we restricted this simulation study to 200 replications per scenario.

Multivariable simulation study

To compare methods in the multivariable setting, we simulated from the same model as in the primary simulation with $J=100$ and $n_X=n_Y$, but where there were two exposures generated according to

$$X_k=\sum _{j=1}^J{\tilde{\beta }}_{Xjk}{G}_j+U+{\epsilon }_{Xk}$$

$$Y=\sum _{k=1}^2{\theta }_k{X}_k+\sum _{j=1}^J{\tilde{\alpha }}_j+U+{\epsilon }_Y\text{,}$$

for $k=\mathrm{1,2}$, where ${\theta }_1=0.1$ or 0, ${\theta }_2=0.1$, and $\left({\epsilon }_{Xk},{\epsilon }_{Xk}\right)$ were simulated from the bivariate normal distribution with marginal variances of 1 and covariance of 0.3.

As well as the multivariable IVW and multivariable IVW-oracle methods, we also compared with the multivariable median approach (MVMR-Median), GRAPPLE, and the constrained maximum likelihood approach with data perturbation (with 100 perturbed samples), which we denote MVMR-cML-DP. Note that for the methods that require the full covariance matrices of the genetic variant-exposures association estimates (that is, GRAPPLE, MVMR-cML-DP, and MVMR-Horse), we input these assuming that the traits are independent, even though they are not. This is to reflect the typical scenario where only GWAS summary statistics are available without trait correlation estimates. Due to the extra computational time required to run the simulation, we restricted this simulation study to 200 replications per scenario.

Applied examples

In a first applied example, we considered combinations of four exposures (LDL cholesterol [LDL], triglycerides [TGs], fasting glucose [FG], and birth weight [BW]) on three outcomes (coronary artery disease [CAD], type 2 diabetes [T2D], and asthma). Each of these exposure-outcome relationships were considered by both Morrison et al.⁶ and Xue et al.,¹⁶ some with conflicting results. Following Morrison et al.,⁶ we classify the exposure-outcome pairs as either having a causal effect supported by the literature (LDL-CAD, TG-CAD, FG-T2D), having unknown or conflicting evidence of a causal effect (BW-CAD, TG-T2D, FG-CAD, BW-T2D, LDL-T2D), or having implausible or unsupported evidence of a causal effect (LDL-Asthma, FG-Asthma, BW-Asthma, TG-Asthma).

We analyzed each of the exposure-outcome pairs using two-sample Mendelian randomization from publicly available GWAS results, as shown in Table 1. All genetic associations were estimated in samples of individuals of European, or predominantly European, ancestry. Note that some of these GWAS datasets differ from those used in Morrison et al.⁶ and Xue et al.,¹⁶ and so there are some differences in the results that we report here.

Table 1.

Sources of genetic association estimates for the exposure and outcome traits considered in the applied examples

Trait	Study	Sample size	Instruments
LDL cholesterol	Willer et al.36	188,577	79
Triglycerides	Willer et al.36	188,577	55
Fasting glucose	Chen et al.37	281,416	70
Birth weight	Horikoshi et al.38	153,781	50
Intelligence	Hill et al.39	248,482	213
Years of education	Okbay et al.40	293,723	213
Household income	Neale lab	361,194	213
Coronary artery disease	Nikpay et al.41	184,305	–
Type 2 diabetes	Mahajan et al.42	898,130	–
Asthma	Demenais et al.43	142,486	–
Alzheimer disease	Lambert et al.44	54,162	–

Instruments is the number of genetic instruments for each exposure trait used in the primary analyses. Genetic association estimates from the Neale lab are sourced from http://www.nealelab.is/uk-biobank/.

For each exposure, we chose a set of instruments by first selecting all genetic variants that associated with the trait with a p value $<5\times {10}^{-8}$, then pruned these to have $r^2<0.001$. The number of selected variants for each exposure is shown in Table 1. The set of variants was then harmonized with each outcome dataset, and only variants that appeared in both exposure and outcome datasets were retained. The F-statistics across each of the 12 exposure-outcome combinations ranged between 51.6 and 172.4. We performed analysis using the IVW, CAUSE, MR-cML, MR-cML-DP, and MR-Horse methods. We used the TwoSampleMR R package⁴⁵ for the pruning and harmonization procedures and the MendelianRandomization R package^46,47 for implementing the IVW, MR-cML, and MR-cML-DP methods.

In a second applied example, we considered a multivariable scenario with conditionally weak instruments. We re-analyzed the example described by Grant and Burgess,³¹ and also considered by Woolf et al.,⁴⁸ which looked at the effects of intelligence, years of education, and household income on the risk of Alzheimer disease. The phenotype referred to here as intelligence is an individual’s score on a verbal-numerical reasoning test used as a proxy for general cognitive function, as described in Hill et al.³⁹ and Davies et al.⁴⁹ The effects of intelligence and years of education on Alzheimer disease were first considered by Davies et al.⁴⁹ using summary statistics from the GWAS of Hill et al.³⁹ (intelligence), Okbay et al.⁴⁰ (years of education), and Lambert et al.⁴⁴ (Alzheimer disease). The genetic instruments were selected as those that associate with either intelligence or years of education with a p value below the genome-wide significance level of $5\times {10}^{-8}$. In Grant and Burgess,³¹ household income was also included using genetic associations for these same genetic variants taken from the UK Biobank sourced from the Neale Lab (http://www.nealelab.is/uk-biobank/). The 213 genetic variants were strong predictors of the exposures separately, with univariable F-statistics all over the conventional threshold of 10. However, the conditional F-statistics were 2.4 (intelligence), 1.6 (years of education), and 1.6 (household income), and so the variants were conditionally weak instruments.

In a supplementary analysis, we repeated this second applied example restricting the set of genetic variants to those that associate with at least one of the three exposures with a p value below $5\times {10}^{-10}$. This approach selected 49 instruments that were separately very strong predictors of the exposures, with univariable F-statistics all over 18. However, the conditional F-statistics were each still quite low (between 1.8 and 2.2) due to the high correlation between the traits.

Results

Simulation study results

The mean estimates, standard deviation of estimates, coverage, and power/type I error rates for each scenario are shown in Table 2 ($J=100$, strong instruments), Table 3 ($J=100$, weak instruments), Table 4 ($J=20$, strong instruments), Table 5 (multivariable setting, showing results relating to estimates of ${\theta }_1$ only), Table S1 (single genetic mechanism), and Table S2 (genome-wide study). For MR-Horse and MVMR-Horse, we computed coverage as the proportion of replications for which the true causal effect was within the 95% credible interval and rejection rate as the proportion of replications for which the 95% credible interval did not contain zero. For the other methods, except for CAUSE, we used the confidence intervals (or credible intervals) produced by their respective software packages to evaluate coverage and rejection rates in the same way (where 95% confidence intervals for BWMR and MR-CUE were produced using the effect estimates and standard errors given by their respective package). For CAUSE, we evaluated coverage using the 95% credible interval and the rejection rate as the proportion of replications for which the one-sided p value (comparing the causal model to the non-causal sharing model) was less than 0.05.

Table 2.

Results of simulations for the scenarios with 100 strong genetic instruments

	20% invalid				40% invalid				60% invalid
Method	Mean	SD	Cov.	Rej.	Mean	SD	Cov.	Rej.	Mean	SD	Cov.	Rej.
$\mathit{\theta }=\mathbf{0.1}$, balanced pleiotropy
IVW	0.156	0.061	0.789	0.807	0.207	0.081	0.673	0.780	0.254	0.093	0.558	0.805
IVW-Oracle	0.098	0.013	0.953	1.000	0.100	0.015	0.953	1.000	0.098	0.019	0.962	0.994
MR-Median	0.107	0.020	0.911	1.000	0.125	0.032	0.737	0.998	0.157	0.059	0.507	0.984
BWMR	0.100	0.017	0.943	0.996	0.196	0.085	0.691	0.768	0.256	0.093	0.560	0.816
MR-cML	0.100	0.014	0.928	1.000	0.101	0.020	0.870	0.999	0.098	0.030	0.774	0.979
MR-cML-DP	0.098	0.014	0.968	1.000	0.099	0.019	0.965	0.999	0.093	0.028	0.948	0.871
MR-Horse	0.100	0.014	0.982	1.000	0.105	0.020	0.987	0.999	0.111	0.030	0.976	0.905
$\mathit{\theta }=\mathbf{0.1}$, directional pleiotropy
IVW	0.158	0.078	0.817	0.627	0.207	0.100	0.778	0.588	0.256	0.117	0.704	0.602
IVW-Oracle	0.098	0.014	0.953	1.000	0.099	0.017	0.945	1.000	0.098	0.021	0.937	0.999
MR-Median	0.108	0.022	0.893	0.999	0.126	0.033	0.729	0.997	0.161	0.065	0.512	0.989
BWMR	0.100	0.016	0.941	1.000	0.166	0.097	0.815	0.745	0.256	0.117	0.707	0.606
MR-cML	0.100	0.015	0.921	1.000	0.102	0.021	0.848	1.000	0.103	0.031	0.773	0.980
MR-cML-DP	0.099	0.015	0.964	1.000	0.100	0.020	0.944	0.997	0.099	0.029	0.942	0.923
MR-Horse	0.101	0.015	0.975	1.000	0.106	0.021	0.981	0.995	0.116	0.031	0.974	0.911
$\mathit{\theta }=\mathbf{0}$, balanced pleiotropy
IVW	0.055	0.061	0.782	0.218	0.106	0.078	0.687	0.313	0.152	0.091	0.573	0.427
IVW-Oracle	0.000	0.013	0.948	0.052	0.000	0.014	0.961	0.039	0.000	0.018	0.954	0.046
MR-Median	0.009	0.020	0.891	0.109	0.024	0.029	0.721	0.279	0.056	0.057	0.502	0.498
BWMR	0.001	0.017	0.935	0.065	0.096	0.084	0.718	0.282	0.153	0.089	0.573	0.427
MR-cML	0.000	0.013	0.925	0.075	0.000	0.018	0.889	0.111	0.000	0.027	0.792	0.208
MR-cML-DP	0.000	0.013	0.975	0.025	0.000	0.017	0.972	0.028	0.000	0.024	0.965	0.035
MR-Horse	0.001	0.014	0.987	0.013	0.004	0.018	0.988	0.012	0.011	0.028	0.977	0.023
$\mathit{\theta }=\mathbf{0}$, directional pleiotropy
IVW	0.060	0.076	0.830	0.170	0.113	0.102	0.747	0.253	0.152	0.119	0.706	0.294
IVW-Oracle	0.001	0.013	0.953	0.047	0.001	0.015	0.955	0.045	0.000	0.018	0.962	0.038
MR-Median	0.011	0.020	0.892	0.108	0.027	0.031	0.702	0.298	0.057	0.058	0.480	0.520
BWMR	0.002	0.015	0.942	0.058	0.065	0.098	0.815	0.185	0.150	0.118	0.712	0.288
MR-cML	0.001	0.014	0.924	0.076	0.003	0.018	0.887	0.113	0.003	0.026	0.808	0.192
MR-cML-DP	0.001	0.013	0.981	0.019	0.003	0.016	0.973	0.027	0.003	0.023	0.974	0.026
MR-Horse	0.002	0.014	0.984	0.016	0.007	0.018	0.993	0.007	0.014	0.026	0.985	0.015

Reported is the mean estimate (mean), standard deviation of estimates (SD), the proportion of replications that the 95% confidence interval (or credible interval in the case of MR-Horse) contained the true causal effect (Cov.), and the proportion of replications that the 95% confidence interval (or credible interval) did not contain zero (Rej.).

Table 3.

Results of simulations for the scenarios with 100 weak genetic instruments

	20% invalid				40% invalid				60% invalid
Method	Mean	SD	Cov.	Rej.	Mean	SD	Cov.	Rej.	Mean	SD	Cov.	Rej.
$\mathit{\theta }=\mathbf{0.1}$, balanced pleiotropy
IVW	0.123	0.054	0.894	0.701	0.160	0.073	0.830	0.674	0.200	0.083	0.726	0.699
IVW-Oracle	0.076	0.013	0.535	1.000	0.076	0.015	0.623	1.000	0.077	0.019	0.735	0.986
MR-Median	0.077	0.020	0.720	0.983	0.086	0.029	0.809	0.958	0.114	0.051	0.797	0.938
BWMR	0.099	0.029	0.916	0.989	0.190	0.098	0.775	0.641	0.254	0.105	0.647	0.698
MR-cML	0.104	0.020	0.883	1.000	0.267	0.433	0.725	0.955	1.313	0.673	0.106	0.959
MR-cML-DP	0.094	0.017	0.978	0.998	0.116	0.140	0.968	0.942	0.717	0.639	0.568	0.727
MR-Horse	0.093	0.017	0.964	0.999	0.093	0.022	0.983	0.957	0.106	0.034	0.988	0.805
$\mathit{\theta }=\mathbf{0.1}$, directional pleiotropy
IVW	0.123	0.068	0.909	0.518	0.168	0.091	0.845	0.509	0.204	0.101	0.824	0.527
IVW-Oracle	0.076	0.012	0.492	1.000	0.077	0.015	0.656	0.998	0.076	0.019	0.742	0.977
MR-Median	0.075	0.019	0.695	0.991	0.088	0.029	0.836	0.962	0.113	0.054	0.815	0.938
BWMR	0.097	0.019	0.931	0.999	0.168	0.111	0.835	0.636	0.255	0.129	0.780	0.506
MR-cML	0.103	0.019	0.891	1.000	0.249	0.452	0.756	0.961	1.392	0.918	0.153	0.939
MR-cML-DP	0.093	0.017	0.985	0.997	0.115	0.154	0.978	0.951	0.634	0.718	0.680	0.691
MR-Horse	0.092	0.016	0.977	1.000	0.095	0.021	0.992	0.963	0.106	0.036	0.988	0.745
$\mathit{\theta }=\mathbf{0}$, balanced pleiotropy
IVW	0.045	0.053	0.828	0.172	0.084	0.069	0.754	0.246	0.120	0.082	0.660	0.340
IVW-Oracle	0.000	0.011	0.946	0.054	0.000	0.013	0.957	0.043	0.000	0.016	0.955	0.045
MR-Median	0.006	0.017	0.936	0.064	0.016	0.025	0.832	0.168	0.038	0.050	0.664	0.336
BWMR	0.001	0.018	0.939	0.061	0.094	0.092	0.783	0.217	0.152	0.103	0.667	0.333
MR-cML	0.000	0.016	0.882	0.118	0.072	0.371	0.789	0.211	0.950	1.186	0.167	0.833
MR-cML-DP	0.000	0.015	0.996	0.004	0.001	0.043	0.994	0.006	0.378	0.693	0.753	0.247
MR-Horse	0.001	0.014	0.986	0.014	0.004	0.019	0.992	0.008	0.011	0.030	0.985	0.015
$\mathit{\theta }=\mathbf{0}$, directional pleiotropy
IVW	0.040	0.067	0.873	0.127	0.079	0.091	0.829	0.171	0.127	0.106	0.739	0.261
IVW-Oracle	0.000	0.011	0.947	0.053	0.001	0.013	0.967	0.033	0.000	0.016	0.964	0.036
MR-Median	0.005	0.017	0.948	0.052	0.015	0.025	0.844	0.156	0.042	0.054	0.643	0.357
BWMR	0.001	0.016	0.945	0.055	0.054	0.103	0.872	0.128	0.157	0.133	0.745	0.255
MR-cML	0.000	0.016	0.878	0.122	0.036	0.349	0.834	0.166	0.969	1.319	0.210	0.790
MR-cML-DP	0.000	0.015	0.990	0.010	0.004	0.061	0.996	0.004	0.326	0.725	0.802	0.198
MR-Horse	0.001	0.014	0.980	0.020	0.005	0.018	0.998	0.002	0.014	0.028	0.988	0.012

Reported is the mean estimate (mean), standard deviation of estimates (SD), the proportion of replications that the 95% confidence interval (or credible interval in the case of MR-Horse) contained the true causal effect (Cov.), and the proportion of replications that the 95% confidence interval (or credible interval) did not contain zero (Rej.).

Table 4.

Results of simulations for the scenarios with 20 strong genetic instruments

	20% invalid				40% invalid				60% invalid
Method	Mean	SD	Cov.	Rej.	Mean	SD	Cov.	Rej.	Mean	SD	Cov.	Rej.
$\mathit{\theta }=\mathbf{0.1}$, balanced pleiotropy
IVW	0.150	0.135	0.897	0.307	0.190	0.178	0.869	0.262	0.244	0.209	0.853	0.277
IVW-Oracle	0.100	0.029	0.965	0.899	0.100	0.034	0.959	0.819	0.099	0.041	0.970	0.610
MR-Median	0.109	0.048	0.919	0.750	0.129	0.086	0.806	0.716	0.198	0.198	0.631	0.730
BWMR	0.135	0.120	0.895	0.456	0.188	0.177	0.870	0.263	0.248	0.207	0.847	0.277
MR-cML	0.101	0.033	0.919	0.905	0.100	0.042	0.874	0.807	0.096	0.061	0.802	0.632
MR-cML-DP	0.097	0.032	0.951	0.806	0.093	0.040	0.935	0.600	0.085	0.053	0.914	0.377
MR-Horse	0.102	0.033	0.989	0.701	0.105	0.044	0.990	0.395	0.122	0.084	0.983	0.211
$\mathit{\theta }=\mathbf{0.1}$, directional pleiotropy
IVW	0.157	0.166	0.902	0.265	0.199	0.221	0.911	0.188	0.261	0.268	0.865	0.230
IVW-Oracle	0.099	0.030	0.957	0.891	0.099	0.035	0.956	0.763	0.097	0.042	0.970	0.568
MR-Median	0.109	0.053	0.893	0.749	0.132	0.093	0.808	0.712	0.201	0.204	0.599	0.713
BWMR	0.134	0.128	0.896	0.528	0.195	0.216	0.885	0.213	0.262	0.265	0.875	0.220
MR-cML	0.101	0.033	0.924	0.909	0.101	0.042	0.876	0.780	0.098	0.059	0.817	0.628
MR-cML-DP	0.097	0.033	0.957	0.830	0.095	0.041	0.934	0.608	0.089	0.053	0.911	0.401
MR-Horse	0.102	0.034	0.983	0.731	0.106	0.044	0.991	0.386	0.122	0.076	0.991	0.183
$\mathit{\theta }=\mathbf{0}$, balanced pleiotropy
IVW	0.046	0.132	0.894	0.106	0.100	0.176	0.882	0.118	0.147	0.204	0.836	0.164
IVW-Oracle	−0.001	0.027	0.969	0.031	0.000	0.032	0.959	0.041	0.000	0.040	0.965	0.035
MR-Median	0.009	0.044	0.915	0.085	0.033	0.093	0.791	0.209	0.093	0.181	0.587	0.413
BWMR	0.032	0.118	0.888	0.112	0.097	0.176	0.877	0.123	0.147	0.204	0.843	0.157
MR-cML	−0.001	0.029	0.939	0.061	0.000	0.039	0.890	0.110	0.000	0.055	0.835	0.165
MR-cML-DP	−0.001	0.027	0.975	0.025	0.000	0.035	0.955	0.045	0.000	0.047	0.942	0.058
MR-Horse	0.000	0.030	0.985	0.015	0.005	0.041	0.994	0.006	0.017	0.071	0.984	0.016
$\mathit{\theta }=\mathbf{0}$, directional pleiotropy
IVW	0.047	0.174	0.883	0.117	0.096	0.227	0.888	0.112	0.153	0.268	0.871	0.129
IVW-Oracle	−0.001	0.028	0.957	0.043	0.001	0.031	0.961	0.039	0.000	0.040	0.963	0.037
MR-Median	0.008	0.048	0.907	0.093	0.031	0.087	0.813	0.187	0.097	0.216	0.591	0.409
BWMR	0.025	0.131	0.886	0.114	0.088	0.219	0.884	0.116	0.152	0.265	0.877	0.123
MR-cML	0.000	0.030	0.932	0.068	0.002	0.037	0.901	0.099	0.003	0.055	0.827	0.173
MR-cML-DP	0.000	0.028	0.969	0.031	0.001	0.034	0.958	0.042	0.002	0.047	0.938	0.062
MR-Horse	0.001	0.031	0.991	0.009	0.006	0.040	0.993	0.007	0.019	0.071	0.993	0.007

Reported is the mean estimate (mean), standard deviation of estimates (SD), the proportion of replications that the 95% confidence interval (or credible interval in the case of MR-Horse) contained the true causal effect (Cov.), and the proportion of replications that the 95% confidence interval (or credible interval) did not contain zero (Rej.).

Table 5.

Results of simulations for the multivariable scenarios (relating to the causal effect of $X_1$ only)

	20% invalid				40% invalid				60% invalid
Method	Mean	SD	Cov.	Rej.	Mean	SD	Cov.	Rej.	Mean	SD	Cov.	Rej.
${\mathit{\theta }}_{\mathbf{1}}=\mathbf{0.1}$, balanced pleiotropy
IVW	0.155	0.066	0.785	0.805	0.204	0.078	0.720	0.765	0.229	0.088	0.670	0.700
IVW-Oracle	0.100	0.014	0.950	1.000	0.098	0.016	0.965	1.000	0.097	0.020	0.970	0.990
MVMR-Median	0.107	0.020	0.935	1.000	0.117	0.027	0.845	1.000	0.137	0.047	0.700	0.975
GRAPPLE	0.109	0.032	0.990	0.775	0.169	0.068	0.875	0.685	0.213	0.088	0.795	0.660
MVMR-cML-DP	0.101	0.015	0.975	1.000	0.099	0.020	0.945	1.000	0.096	0.031	0.935	0.850
MVMR-Horse	0.102	0.015	0.980	1.000	0.105	0.020	1.000	0.990	0.114	0.032	0.985	0.880
${\mathit{\theta }}_{\mathbf{1}}=\mathbf{0.1}$, directional pleiotropy
IVW	0.154	0.077	0.855	0.620	0.208	0.109	0.790	0.585	0.229	0.105	0.805	0.535
IVW-Oracle	0.098	0.014	0.955	1.000	0.100	0.017	0.965	1.000	0.098	0.022	0.940	0.990
MVMR-Median	0.105	0.019	0.970	1.000	0.119	0.032	0.820	0.995	0.142	0.051	0.670	0.965
GRAPPLE	0.109	0.031	0.995	0.680	0.175	0.087	0.900	0.510	0.217	0.103	0.830	0.490
MVMR-cML-DP	0.100	0.016	0.975	1.000	0.103	0.020	0.955	0.990	0.106	0.030	0.950	0.935
MVMR-Horse	0.101	0.015	0.975	1.000	0.109	0.021	0.985	0.995	0.121	0.034	0.970	0.870
${\mathit{\theta }}_{\mathbf{1}}=\mathbf{0}$, balanced pleiotropy
IVW	0.055	0.061	0.770	0.230	0.103	0.078	0.690	0.310	0.138	0.086	0.705	0.295
IVW-Oracle	−0.001	0.013	0.975	0.025	0.003	0.017	0.925	0.075	−0.004	0.020	0.950	0.050
MVMR-Median	0.004	0.017	0.960	0.040	0.020	0.028	0.775	0.225	0.030	0.042	0.750	0.250
GRAPPLE	0.006	0.027	1.000	0.000	0.070	0.068	0.835	0.165	0.118	0.083	0.775	0.225
MVMR-cML-DP	−0.001	0.014	0.970	0.030	0.002	0.020	0.955	0.045	−0.008	0.028	0.930	0.070
MVMR-Horse	0.000	0.014	1.000	0.000	0.008	0.022	0.995	0.005	0.007	0.030	0.980	0.020
${\mathit{\theta }}_{\mathbf{1}}=\mathbf{0}$, directional pleiotropy
IVW	0.049	0.072	0.870	0.130	0.096	0.104	0.795	0.205	0.133	0.115	0.810	0.190
IVW-Oracle	0.000	0.014	0.940	0.060	−0.002	0.017	0.960	0.040	−0.001	0.021	0.960	0.040
MVMR-Median	0.006	0.020	0.945	0.055	0.014	0.030	0.830	0.170	0.031	0.041	0.770	0.230
GRAPPLE	0.011	0.028	1.000	0.000	0.059	0.083	0.915	0.085	0.112	0.111	0.840	0.160
MVMR-cML-DP	0.001	0.016	0.950	0.050	0.000	0.020	0.955	0.045	0.002	0.031	0.910	0.090
MVMR-Horse	0.002	0.016	0.975	0.025	0.005	0.022	0.990	0.010	0.014	0.031	0.965	0.035

Reported is the mean estimate (mean), standard deviation of estimates (SD), the proportion of replications that the 95% confidence interval (or credible interval in the case of MVMR-Horse) contained the true causal effect (Cov.), and the proportion of replications that the 95% confidence interval (or credible interval) did not contain zero (Rej.).

When genetic instruments were strong and explained a large proportion of the variance in the exposure, the mean estimates from MR-cML, MR-cML-DP, and MR-Horse were close to IVW-Oracle at all levels of pleiotropy. The IVW and BWMR methods showed considerable bias from 40% pleiotropy, as did MR-Median at 60% pleiotropy. Only MR-cML-DP and MR-Horse retained nominal coverage and type I error rates in all scenarios. In general, MR-cML-DP showed slightly higher power, whereas MR-Horse showed slightly higher coverage rates. These results were similar in the case where this setting was repeated but with genetic variants acting on the exposure only via a single mechanism. Furthermore, MR-CUE, which assumes the single mechanism and allows for correlated pleiotropy, showed substantial bias and inflated type I error rates in these settings.

When the genetic instruments were weak, the estimates from IVW and IVW-Oracle attenuated to the null, which is the expected behavior of these methods with weak instruments. MR-Median and BWMR showed bias and inflated type I error rates across all medium to high-level pleiotropy settings. MR-Median does not account for the uncertainty in the genetic variant-exposure association estimates and so is not robust to weak instruments. BWMR does account for this uncertainty, but does not allow for InSIDE violations. MR-cML also showed considerable bias and inflated type I error rates in the medium to high-pleiotropy settings, as did MR-cML-DP in the high-pleiotropy settings. On the other hand, MR-Horse had low levels of bias and had type I error rates below the nominal level in all scenarios.

One possible explanation for the performance of the constrained maximum likelihood methods with weak and pleiotropic instruments is the difficulty in selecting the invalid instruments in these settings. These approaches rely on a parameter (which we denote $\tilde{K}$) representing the number of invalid instruments, which determines the constraint placed on the likelihood maximization. Xue et al.¹⁶ showed that the methods will asymptotically estimate the correct value of $\tilde{K}$ but noted that this consistency in selection may not be obtained when there are weak and pleiotropic instruments. Here, we implemented their recommended model averaging approach, which fits the model for all possible values of $\tilde{K}$ and computes a weighted average of the effect estimates with weights determined by the Bayesian information criterion for each fit. In a further supplementary simulation study, we re-analyzed the simulated data from the weak instruments scenario but fixed the values of $\tilde{K}$ instead of using model averaging. We set the value of $\tilde{K}$ at 20, 40, and 60 for each setting regardless of the true level of pleiotropy. The results are shown in Tables S3 and S4. Setting the value of $\tilde{K}$ to its true value substantially reduced the bias from these methods; however, the bias was still greater than that of MR-Horse in the non-null settings. Furthermore, MR-cML still had inflated type I error rates, and MR-cML-DP had very low power. Setting the value of $\tilde{K}$ conservatively to 60 led to very low power when the true number of invalid instruments was below 60. On the other hand, fixing $\tilde{K}$ lower than the true value (for example, setting $\tilde{K}=20$ when the true value was 40 or 60) resulted in substantial bias.

When the genetic instruments were strong but explained a low proportion of the variation in the exposure, the performances of each method were broadly similar to in the first setting but with lower power across the board. At the high pleiotropy levels, MR-cML-DP had type I error rates above the nominal level, whereas MR-Horse did not. In the genome-wide simulation study, MR-Horse had coverage and bias comparable to CAUSE, with substantially higher power in the causal settings. It again had slightly higher coverage and lower type I error rates compared with MR-cML-DP and outperformed all other methods across each metric. In the multivariable simulation study, MVMR-Horse outperformed IVW, MVMR-Median, and GRAPPLE in terms of bias, precision, and type I error rates in all scenarios. Similar to the univariable settings, MVMR-Horse retained type I error rates below the nominal level in all scenarios, with the trade-off of lower power compared with MVMR-cML-DP. MVMR-cML-DP had inflated type I error rates in the higher pleiotropy scenarios.

Overall, the results from the simulation study show that MR-Horse and MVMR-Horse have levels of performance similar to MR-cML-DP and MVMR-cML-DP when genetic instruments are strong and explain a high amount of the variation in the exposure. However, particularly at high levels of pleiotropy, MR-Horse outperforms MR-cML-DP when instruments are weak and/or explain only a small amount of variation in the exposure. Although conservative, MR-Horse retained low bias and type I error rates below the nominal level in all scenarios considered and was the only method to do so out of those considered.

Applied examples results

For the univariable analyses, Figure 2 shows the estimated log odds ratio of each outcome per standard deviation increase in the genetically predicted levels of each exposure, as well as the 95% confidence intervals (IVW, MR-cML, MR-cML-DP) or 95% credible intervals (CAUSE, MR-Horse). For the exposure-outcome combinations with established causality, the results from each method were broadly in line across the different methods. For LDL-T2D, CAUSE did not show evidence of a causal effect. MR-Horse agreed with MR-cML(-DP) that increased LDL may reduce T2D risk, although with a 95% credible interval that came closer to the null. The other conflicting result was that CAUSE and MR-cML showed evidence of a causal effect of TGs on T2D risk, whereas MR-cML-DP and MR-Horse produced a null result. We note, however, that the one-sided p value from the CAUSE test comparing the causal model to the non-causal model was 0.066. Because MR-cML-DP and MR-Horse showed lower type I error rates in our simulation studies, one possible explanation for this difference is that MR-cML may be producing a false-positive result in this case. All methods showed a causal effect of BW on the risk of CAD and T2D, though with the estimates from CAUSE much closer to the null than the others. Finally, all methods gave a null result for the causal effect of LDL, TGs, and BW on the risk of asthma. However, MR-Horse showed a possible causal relationship between fasting glucose and risk of asthma, agreeing with MR-cML(-DP).

Figure 2.

Mendelian randomization estimates and 95% confidence intervals (IVW, MR-cML, MR-cML-DP) or 95% credible intervals (CAUSE, MR-Horse) for the different exposure-outcome combinations

Plots colored in orange indicate exposure-outcome relationships considered causal. Plots colored in yellow indicate exposure-outcome pairs with unknown causal relationships. Plots colored in blue indicate exposure-outcome relationships considered not causal. In each plot, the vertical dashed line indicates a log odds ratio of zero.

For the multivariable analysis with weak instruments, Table 6 shows the estimated log odds ratio of Alzheimer disease, as well as the 95% confidence intervals or credible intervals, from the multivariable IVW, MVMR-Median, MVMR-cML, MVMR-cML-DP, and MVMR-Horse methods. The reported estimates are per one standard deviation increase in genetically predicted intelligence and years of education and per unit increase in log odds ratio for a higher household income bracket. For the effect of intelligence on Alzheimer disease, the estimates were broadly in line across each method; however, MVMR-cML-DP had a very wide confidence interval. For the effects of years of education and household income, MVMR-cML and MVMR-cML-DP showed substantially different point estimates compared to the other methods and very wide confidence intervals. On the other hand, the effect estimates from MVMR-Horse were similar to those from MVMR-Median, though with an estimate in the opposite direction for years of education. These results from MVMR-Horse, including the directions of effects, are similar to the effect estimates reported in Grant and Burgess³¹ using the MVMR-Lasso method, but with wider confidence intervals (noting that MVMR-Lasso has been shown to produce unbiased effect estimates but with confidence intervals that are too narrow).

Table 6.

Mendelian randomization estimates for the effects of intelligence, years of education, and household income on Alzheimer disease using multivariable models

Method	Intelligence	Education	Income
IVW	−0.47 (−0.86, −0.07)	−0.24 (−0.92, −0.43)	0.42 (−0.25, −1.08)
MVMR-Median	−0.57 (−1.05, −0.10)	−0.13 (−0.89, −0.62)	0.37 (−0.39, −1.12)
MVMR-cML	−0.35 (−1.01, −0.32)	−2.90 (−4.92, −0.87)	3.52 (−1.58, −5.46)
MVMR-cML-DP	−0.73 (−2.12, −0.66)	−0.39 (−5.27, −4.50)	1.08 (−3.52, −5.68)
MVMR-Horse	−0.74 (−1.32, −0.15)	−0.12 (−0.81, −1.06)	0.46 (−0.49, −1.39)
Univariable F-statistic	31.4	21.0	10.5
Conditional F-statistic	2.4	1.6	1.6

Reported are the estimated log odds ratios of Alzheimer disease and 95% confidence intervals (IVW, MVMR-Median, MVMR-cML, MVMR-cML-DP) or 95% credible intervals (MVMR-Horse). Also reported are the univariable and conditional F-statistics for each exposure.

The results of the multivariable analysis re-analyzed with unconditionally stronger instruments are shown in Table S5. Due to the reduced number of instruments, the confidence intervals and credible intervals are much wider than the original analysis. Although there are some differences in effect estimates, the confidence intervals and credible intervals are overlapping with the original ones, and the results from IVW, MVMR-Median, and MVMR-Horse are still broadly in line with each other. The results from MR-cML and MR-cML-DP are still producing substantially different effect estimates to the other results, with particularly wide confidence intervals.

Computational run time

In order to give an idea of the comparative computational load of the proposed methods, we measured the run time of the applied analyses on a machine running an Apple M1 Pro chip. To perform all 12 univariable analyses in the first applied example (with between 19 and 78 genetic instruments), MR-Horse took 141 s, compared with MR-cML-DP, which took 201 s. The run time for MVMR-Horse increased with multiple exposures and larger numbers of genetic variants. In the second applied example, the analysis with 3 exposures and 213 genetic variants took 996 s with MVMR-Horse compared to 220 s with MVMR-cML-DP.

Discussion

In this paper, we have presented a Bayesian framework for performing Mendelian randomization with summary-level data that is robust to general forms of instrument invalidity. The approach does not require the InSIDE assumption, and so it can handle correlated pleiotropy. The method is easily generalized to the multivariable Mendelian randomization setting, providing another robust method for multivariable studies. Our simulation studies showed that the approach has favorable performance in comparison with commonly used methods across a variety of settings.

Our approach has similarities to previously proposed Bayesian approaches but has been designed to avoid many of the restrictions that these methods pose. Berzuini et al.¹⁷ first proposed a Bayesian framework for Mendelian randomization with pleiotropy and suggested the use of the horseshoe prior to account for the presence of some valid and some invalid instruments. However, their method requires individual-level data and makes the InSIDE assumption. Zhao et al.¹⁸ and Bucur et al.¹⁹ developed versions of the Berzuini et al. approach using summary-level data, again requiring InSIDE to be satisfied. Cheng et al.²⁰ introduced a correlation parameter into a Bayesian model to account for correlated pleiotropy; however, their model requires that pleiotropic effects are either all correlated or all uncorrelated with the variant-exposure effects. The methods of Morrison et al.⁶ and Cheng et al.²¹ are able to account for both correlated and uncorrelated pleiotropy, but they make a further assumption that the genetic instruments are associated with the exposure either entirely through the confounders or entirely through a direct effect. Our approach utilizes the horseshoe prior as suggested by Berzuini et al.,¹⁷ can be applied using summary-level data, and can account for both correlated and uncorrelated pleiotropy without any assumption on the pathways between the genetic instruments and exposure. Furthermore, we have shown how to extend the framework to be used for multivariable Mendelian randomization.

In comparison to frequentist approaches for Mendelian randomization, our method is based on the same causal model as that used by Xue et al.,¹⁶ which we believe is a very general model for instrument invalidity. With strong genetic instruments, the performance of our method in simulation studies was comparable to Xue et al.’s method when the latter was applied with the data perturbation extension in scenarios with low to mid-levels of pleiotropy. However, when the genetic instruments were weak and/or explained a small amount of variation in the exposure, MR-Horse outperformed all other methods considered. In practice, Mendelian randomization studies with weak instruments, both in the sense of low F-statistics and low proportion of variation in the exposure explained, are common, particularly with the sort of complex exposures that are being increasingly studied with this design. It is also likely that, particularly when many genetic variants are selected as instruments from high-powered GWASs, the proportion of invalid instruments is similar to the high pleiotropy scenarios considered in our simulation study. Thus, the simulation scenarios whose results are shown in Table 3 (weak instruments) and Table 4 (low proportion of variation explained) are likely to closely reflect many real-world settings. MR-Horse was the only method considered that retained low levels of bias and that showed coverage and type I error rates at their nominal levels in these cases.

The trade-off to the Bayesian approach retaining low type I error rates is that it sometimes had lower power than the frequentist approaches. The increased conservatism in the Bayesian approach supports the assertion that the horseshoe prior is able to avoid shrinkage of the pleiotropic effects pertaining to invalid instruments. Besides improved type I error control, another advantage of our framework is that it allows for the model specification to be adapted to specific settings. For example, the prior distribution of the global shrinkage parameter could be adjusted, or set at a fixed value, based on prior knowledge of the level and nature of pleiotropy. As long as these prior assumptions are reasonable, such incorporation of prior knowledge would be expected to reduce the uncertainty in the estimates and increase power.

Although the implementation of the method in this paper has used some weakly informative priors for the model parameters, one case where we have used a more informative prior is for the ${\rho }_j$ parameters that model the correlation between the genetic variant-exposure associations and any pleiotropic effects. The justification for using a prior that places the majority of the mass close to zero is that this is consistent with our a priori assumption that some genetic instruments are valid and some are invalid. Those that are valid will have a true value of ${\rho }_j$ of zero, as there are no pleiotropic effects. Relaxing this prior to, say, a uniform distribution on $\left(-\mathrm{1,1}\right)$ would allow for all variant-exposure associations to have potentially substantial correlation; however, this would come at a cost of lower power. On the other hand, this prior could be strengthened by adding even more weight close to zero, possibly in combination with a stricter prior on the global shrinkage parameter, as discussed above. An interesting area for future development of the approach would be to explore the use of domain knowledge to further inform the model inputs.

Not taken into account in our model are sample overlap and linkage disequilibrium. Most of the methods that are commonly used for Mendelian randomization assume independent genetic variants, and they achieve this by pruning candidate instruments according to an $r^2$ threshold. In general, the use of correlated variants is unlikely to add substantially more information to an analysis, and methods that are able to incorporate linkage disequilibrium tend to be very sensitive to mis-specification of genetic correlations.⁵⁰ Nonetheless, there are situations where correlated variants may be desirable, such as in a cis-Mendelian randomization study exploring drug targets.⁵¹ In this case, correlation estimates could be incorporated into our model by specifying a joint distribution for the genetic variant-exposure association estimates and for the genetic variant-outcome association estimates that included genetic correlations, as is done by Cheng et al.²¹ to account for linkage disequilibrium.

Sample overlap is more likely to be an issue in practice, since many published GWAS results are from meta-analyses that include common data sources such as the UK Biobank.⁵² The models of Morrison et al.⁶ and Cheng et al.²¹ account for sample overlap by including correlation between ${\hat{\beta }}_{Yj}$ and ${\hat{\beta }}_{Xj}$ in a joint distribution. Our model could be adapted in a similar way to account for sample overlap if it was believed to be substantial.

Assumptions that we have made are of the linearity of the effect of the exposures on the outcome, and homogeneity or no effect modification. Furthermore, an implicit assumption that we have made in developing the method and simulation study is that all traits are continuous. In practice, summary-level Mendelian randomization methods can be performed with categorical traits, for example, by taking genetic association estimates from logistic regression. This may bias causal effect estimates because of the non-collapsibility of odds ratios; however, the bias will tend to be toward the null, at least in the univariable case.⁵³ One further drawback to Bayesian methods generally is that they tend to be computationally intensive. However, in the univariable case, MR-Horse actually ran the applied analyses more quickly than MR-cML-DP. In the multivariable case, MVMR-Horse took longer to run the applied analyses than MVMR-cML-DP; however, the run time was not prohibitively slow, even with a fairly large set of genetic instruments.

Overall, our Bayesian framework is an effective and flexible method for both univariable and multivariable Mendelian randomization. It provides an important sensitivity analysis to these studies that is robust to violations of the InSIDE assumption and directional pleiotropy. Furthermore, it can be easily adapted to handle sample overlap or linkage disequilibrium.

Data and code availability

All the data used in this paper are publicly available and can be accessed via the references given. R code for performing the proposed methods, and for reproducing the simulation results and applied analyses, can be found at https://github.com/aj-grant/mrhorse.

Published: January 4, 2024