Selecting Invalid Instruments to Improve Mendelian Randomization with Two-Sample Summary Data

Abstract

Mendelian randomization (MR) is a widely-used method to estimate the causal relationship between a risk factor and disease. A fundamental part of any MR analysis is to choose appropriate genetic variants as instrumental variables. Genome-wide association studies often reveal that hundreds of genetic variants may be robustly associated with a risk factor, but in some situations investigators may have greater confidence in the instrument validity of only a smaller subset of variants. Nevertheless, the use of additional instruments may be optimal from the perspective of mean squared error even if they are slightly invalid; a small bias in estimation may be a price worth paying for a larger reduction in variance. For this purpose, we consider a method for “focused” instrument selection whereby genetic variants are selected to minimise the estimated asymptotic mean squared error of causal effect estimates. In a setting of many weak and locally invalid instruments, we propose a novel strategy to construct confidence intervals for post-selection focused estimators that guards against the worst case loss in asymptotic coverage. In empirical applications to: (i) validate lipid drug targets; and (ii) investigate vitamin D effects on a wide range of outcomes, our findings suggest that the optimal selection of instruments does not involve only a small number of biologically-justified instruments, but also many potentially invalid instruments.

1. Introduction

Mendelian randomization (MR) uses genetic variants as instrumental variables to estimate the causal effect of a risk factor on an outcome in the presence of unobserved confounding. By Mendel’s second law, genetic variants sort independently of other traits. Thus, genetic variants, which are fixed at conception, can provide a source of exogenous variation in a risk factor of interest, allowing analyses that are less vulnerable to reverse causality and confounding (Davey Smith and Ebrahim, 2003).

Large-scale consortia genome-wide association studies (meta-GWASs) have identified large numbers of genetic variants that are robustly associated with a wide range of traits. Due in part to privacy issues, often only summary statistics of these genetic associations are made publicly available. Since these results are easily accessible, MR investigations increasingly rely on inferential methods that require only two-sample summary data (Burgess et al., 2015). In such applications, genetic variant associations with the risk factor are obtained from a representative but non-overlapping sample used to measure genetic variant associations with the outcome.

As in usual instrumental variable analyses, identifying causal effects through MR requires some key assumptions. For a genetic variant to be a valid instrument, it must be associated with the risk factor; β_Xj ≠ 0 in Figure 1 (relevance). Second, the variant must be uncorrelated with unobserved confounders. Third, any effect that the variant has on the outcome must be mediated by its effect on the risk factor; τ_j = 0 in Figure 1 (exclusion).

Fig 1. The effect of genetic variant Z_j on the risk factor X and outcome Y, where U is an unobserved confounder.

Violations of the exclusion condition are common in MR studies (Lawlor et al., 2008), due in part to the widespread phenomenon of pleiotropy where a single genetic variant may influence several traits (Solovieff et al., 2013; Hemani, Bowden and Davey Smith, 2018). While the biological mechanism of certain genetic effects may be well understood (for example, when using protein risk factors for drug target validation; see Schmidt et al., 2020), it is generally difficult to rule out the possibility that many genetic variants have a direct effect on the outcome, and thus violate the exclusion restriction (Verbanck et al., 2018).

In some MR applications, investigators may have more confidence in the validity of a particular subset of all candidate instruments. In particular, the causal mechanisms linking specific genes to risk factors may be known, which may better justify the use of genetic variants from those genes as instruments. Examples of MR studies that have prioritised the use of variants from biologically-plausible genes include investigations into the effects of alcohol consumption (Millwood et al., 2019), C-reactive protein level (Swerdlow et al., 2016), smoking behaviour (Lassi et al., 2016), vitamin D supplementation (Mokry et al., 2015), and perturbing drug targets (Gill et al., 2021).

Along with biologically-justified instruments, investigators could consider using additional variants that are plausibly valid instruments in order to improve the precision of an analysis. Even if these additional instruments are slightly invalid, their use may provide slightly biased but more precise estimates that are optimal from the perspective of mean squared error.

Hence, to guide this instrument choice, we propose a method for “focused” instrument selection whereby genetic variants are selected to minimise the estimated asymptotic mean squared error of causal effect estimates. The strategy allows a tiering in the assumptions on instrument validity, and prioritises the evidence suggested by biologically-justified instruments.

We work with the popular two-sample summary data design, and in a setting of many weak and locally invalid instruments. In this local mis-specification setting, the collective direct instrument effects on the outcome are decreasing with the sample size at a rate that ensures a meaningful bias-variance trade off, and thus enables a mean squared error comparison in finite samples.

The theoretical contribution of our work has two elements. First, we extend DiTraglia (2016)’s results to allow for many weak instruments. We consider an asymptotic framework in which the number of instruments can grow at the same rate as the sample size, as long as their collective effects on the risk factor are bounded (Zhao et al., 2020). Second, we consider the problem of post-selection inference for focused estimators which is particularly challenging because we do not have consistent model selection; the uncertainty in model selection directly impacts the asymptotic distribution of focused estimators.

Given that focused use of additional instruments may lead to improved estimation compared with using only a smaller set of biologically-justified instruments, it is natural to consider whether a similar advantage may hold for inference. One desirable property for confidence intervals is that they are uniformly valid; that is, they achieve nominal coverage asymptotically over the space of potential direct instrument effects on the outcome. Unfortunately, the impossibility results discussed by Leeb and Pötscher (2005) suggest we cannot construct uniformly valid confidence intervals for focused estimators which will exactly achieve nominal coverage, and therefore such intervals will generally be conservative. For example, in simulation we find that the “2-step” uniformly valid confidence intervals of DiTraglia (2016) can be over 30% longer than standard confidence intervals based on using only a core set of valid instruments.

In the same way that focused estimation is willing to trade a small bias in estimation for a larger reduction in variance, we develop a strategy for constructing “focused” confidence intervals that accepts a small loss in coverage probability over a subspace of direct instrument effects on the outcome in exchange for shorter confidence intervals. Our focused confidence intervals account for uncertainty in model selection, and they achieve nominal coverage for plausible values of direct instrument effects, while providing a statistical guarantee on the worst case size distortion away from those values.

Compared with using only a core set of biologically-justified instruments, our simulation evidence illustrates how focused estimators and confidence intervals may provide improved estimation and inference when additional instruments are valid or slightly invalid, while buying insurance against poor performance when additional instruments are very invalid.

The utility of our methods is demonstrated in two empirical applications. The first considers genetic validation of lipid drug targets when investigators are uncertain about defining the width of a cis-gene window from which to select instruments. The second application investigates the genetically predicted effect of vitamin D supplementation on a range of outcomes when investigators want to prioritise the use of genetic variants from specific genes through biological considerations.

We use the following notation and abbreviations: $\underrightarrow{P}$ ‘converges in probability to’; $\underrightarrow{D}$ ‘converges in distribution to’; $\underset{~}{a}$ ‘is asymptotically distributed as’. For any sequences a_n and b_n, if a_n = O(b_n), then there exists a positive constant M and a positive integer N such that for all n ≥ N, b_n > 0 and |a_n| ≤ Mb_n. If a_n = o(b_n), then |a_n|/b_n → 0 as n → ∞. Also, if a_n = Θ(b_n), then there exist positive constants M₁ and M₂, M₁ ≤ M₂ < ∞, and a positive integer N such that M₁b_n ≤ a_n ≤ M₂b_n for all n ≥ N. The proofs of theoretical results are given in Supplementary Material (Patel et al., 2024), and R code to perform our empirical investigation is available on GitHub at github.com/ash-res/focused-MR/.

2. Model and assumptions

2.1. Two-sample summary data

We first outline our assumptions on genetic association summary data, which are motivated through a simple linear model with invalid instruments and homoscedastic errors.

Let Z = (Z₁,…, Z_p)′ denote a p-vector of uncorrelated genetic variants. The parameter of interest is the causal effect θ₀ of the risk factor X on the outcome Y, which is described by

$$Y={\omega }_Y+X{\theta }_0+Z^{\prime }\tau +U_Y$$ (1)

$$X={\omega }_X+Z^{\prime }{\beta }_X+U_X,$$ (2)

where $E\left[U_Y|Z\right]=0,E\left[U_X|Z\right]=0,E\left[U_Y^2|Z\right]={\sigma }_{U_Y}^2,E\left[U_X^2|Z\right]={\sigma }_{U_X}^2,{\beta }_X={\left({\beta }_{X_1},\mathrm{\dots },{\beta }_{X_p}\right)}^{\prime }$, and (ω_X, ω_Y, β_X, τ, θ₀) are unknown parameters. If τ is non-zero, then at least one genetic variant fails the exclusion restriction and directly affects the outcome.

Substituting (2) into (1), we have Y = (ω_Y + ω_Xθ₀) + Z′(β_Xθ₀ + τ) + (U_Y + U_Xθ₀). Thus, Cov(Z, Y) = Var(Z)(β_Xθ₀ + τ), which leads to a model

$${\beta }_Y={\beta }_X{\theta }_0+\tau ,$$ (3)

where β_Y = (β_Y1,…, β_Yp)′ is the p-vector of coefficients from a population regression of Y on Z. We aim to estimate the model in (3) using two-sample summary data on genetic associations.

Assumption 1 (two-sample summary data)

For each variant j, we observe genetic associations ${\widehat{\beta }}_{X_j}$ and ${\widehat{\beta }}_{Y_j}$, which satisfy ${\widehat{\beta }}_{X_j}~N\left({\beta }_{X_j},{\sigma }_{X_j}^2\right)$ and ${\widehat{\beta }}_{Y_j}~N\left({\beta }_{Y_j},{\sigma }_{Y_j}^2\right)$, where ${\sigma }_{X_j}^2=\Theta \left(1/n_X\right)$ and ${\sigma }_{Y_j}^2=\Theta \left(1/n_Y\right)$ are assumed to be known. Moreover, the set of 2p genetic associations ${\{{\widehat{\beta }}_{X_j},{\widehat{\beta }}_{Y_j}\}}_{j=1}^p$ are mutually uncorrelated, and n_X/n_Y → c, as n := (n_X, n_Y) → ∞ for some constant 0 < c < ∞.

Assumption 1 is taken from Zhao et al. (2020) and states a normal approximation of estimated genetic associations which is typically justified by large random sampling expected in genetic association studies. Specifically, for each variant j, we have access to estimates and standard errors from univariable X on Z_j linear regressions from an n_X-sized sample, and from a non-overlapping n_Y-sized sample, we observe measured associations from univariable Y on Z_j linear regressions. Both random samples are drawn from the joint distribution of (Y, X, Z).

The assumption that the population standard deviations ${\left\{{\sigma }_{X_j},{\sigma }_{Y_j}\right\}}_{j=1}^p$ are known is common in MR, and a formal justification for this is given by Ye, Shao and Kang (2021). To simplify notation, the dependence of the standard errors on the sample size is not made explicit, but they are assumed to decrease at the usual parametric rate.

2.2. Core instruments

From (3), the parameter of interest θ₀ is not identified unless there are some restrictions on τ = (τ₁,…, τ_p)′. When genetic variants from several gene regions are used to instrument the risk factor, a popular identification strategy is to assume that τ is a mean zero random effect; see, for example, Zhao et al. (2020). Another commonly-used assumption is that most genetic variants are valid instruments; τ_j = 0 for j ∈ S_M, where S_M is some unknown set of variants such that p⁻¹|S_M| > 0.5. In this setting the median-based estimators of Bowden et al. (2016) are consistent. Based on a variation of these assumptions, many summary data MR methods have proposed ways to obtain unbiased estimates of θ₀; a recent review is given in Sanderson et al. (2022).

In this work we do not focus on a novel identification strategy, but instead only the simple instrument selection choice for investigators when they believe a core set of genetic variants S₀ consists of only valid instruments, but they are less confident on the validity of any additional candidate instruments.

Assumption 2 (core instruments)

For some known set of instruments S₀ such that 1 ≤ |S₀| < p, and where |S₀| grows proportionately with p, we have τ_j = 0 for all j ∈ S₀.

Assumption 2 allows us to measure the bias which may result from the inclusion of additional instruments. In some MR studies, there is good reason for believing that a smaller subset of all available genetic variants are more likely to be valid instruments. We mention two examples that we explore in more detail in Section 6.

Example 1 (choosing cis windows in drug target MR)

In drug target MR studies, only those genetic variants from a single gene region that encodes the protein target of a drug are used to instrument the risk factor. Such studies have gained popularity for providing supporting genetic evidence to validate drug targets and to study side effects (Gill et al., 2021). A key decision in drug target MR is to choose the width of a “cis window” which dictates a gene region from which to select instruments. Tools such as GeneCards (Stelzer et al., 2016) offer practical guidance on defining appropriate gene regions, but since there are often relatively few uncorrelated genetic signals from narrow cis windows, researchers often resort to widening the cis window in order to boost power through the use of a larger number of instruments. This leaves the possibility of a type of publication bias where researchers may report findings only from a cis window which gives their preferred result. We may be less confident on the instrument validity of additional genetic variants that are included only from widening a cis window.

Example 2 (estimating vitamin D effects)

GWASs have identified strong genetic associations with vitamin D in biologically plausible genes; GC, DHCR7, CYP2R1, and CYP24A1. Each of these genes is known influence vitamin D level through different mechanisms. In order to investigate the effect of vitamin D supplementation on a range of traits and diseases, MR studies have often used genetic variants located in neighborhoods of those genes to instrument vitamin D as they are considered more likely to satisfy the exclusion restriction than other genome-wide significant variants (Mokry et al., 2015; Revez et al., 2020). Genetic variants from other gene regions may be strongly associated with vitamin D, but they may be more likely to have direct effects on the outcome through their effects on traits other than vitamin D.

2.3. Many weak variant associations

In typical applications, we may expect many genetic variants to have weak effects on the risk factor. This can cause difficulties for identifying and estimating the causal effect. We study a setting of many weak instruments where the number of genetic variants is permitted to grow at the same rate as the sample size, but their collective explanatory power is bounded.

Assumption 3 (many weak instruments)

Let β_X,0 denote a |S₀|-vector with its elements given by β_Xj for j ∈ S₀. Then, ||β_X||₂ = O(1), ||β_X||₃/||β_X,0||₂ → 0, and $p/n{\Vert {\beta }_{X,0}\Vert }_2^2=O\left(1\right)$ as n, p → ∞.

Given (3) and Assumptions 2 and 3, the causal effect is point identified. Assumption 3 is similar to Zhao et al. (2020), and it implies that all variants are relevant instruments, but the explanatory power of any individual variant is decreasing as p → ∞. Moreover, the skewness of β_X is restricted which rules out very sparse variant effects settings. Finally, the rate restriction $p/n{\Vert {\beta }_{X,0}\Vert }_2^2=O\left(1\right)$ as n, p → ∞ ensures asymptotic normality in estimation, and the condition is plausible given the large sample sizes of typical GWASs.

2.4. Additional locally invalid instruments

To meaningfully study a bias-variance trade off from including additional instruments we need to ensure that the bias and variance terms have the same order of magnitude. Following DiTraglia (2016)’s framework of locally invalid instruments, we work with local mis-specification in which the direct effects τ are collectively “local-to-zero”, and decrease at the same rate as the sampling errors with n. This is not a substantive biological assumption, but rather a technical device that allows us to study the instrument selection problem from a mean squared error perspective.

Assumption 4 (locally invalid instruments)

${\Vert \tau \Vert }_2^2=O\left(1/n\right)$ as n, p → ∞.

Under the rate restriction in Assumption 4, we can consistently estimate θ₀ using any set of instruments, but making valid inferences using invalid instruments will require us to account for an asymptotic bias. The rate restriction is slightly different to DiTraglia (2016) in that we limit only the collective direct effects over all variants.

Assumption 4 is otherwise quite general. For example, the direct effects τ may be correlated with β_X, and thus may violate the so-called InSIDE (Instrument Strength Independent of Direct Effect) assumption which some existing MR methods that make use of invalid instruments rely on (Bowden, Smith and Burgess, 2015).

3. Focused instrument selection and estimation

Using only the core set of instruments S₀ should lead to asymptotically unbiased estimation of θ₀, and this forms a basis from which to measure the potential bias from including additional instruments. For ease of exposition we focus on the simple choice between using either: (i) the “Core” estimator that chooses only the set of core instruments S₀; or (ii) the “Full” estimator that uses the full set of p instruments. The inclusion of S, the set of the additional p – |S₀| genetic variants, may result in improved estimation if there is a large reduction in variance, and at most only a small increase in bias. Thus we use a “focused” instrument selection strategy (DiTraglia, 2016) where the Full estimator is selected only if it has a lower estimated asymptotic mean squared error than the Core estimator.

The results presented here can be easily extended for more fine-tuned instrument selection at the expense of extra notation; instead of simply choosing between the Core and Full estimators, we could also select from subsets of the additional instruments. Such subsets could be data-driven, chosen on biological reasons, and have overlapping variants. For example, in our empirical application to study vitamin D effects, we chose from all possible subsets of 3 partitions of the additional instruments, where the partitions were formed by k-means clustering on the Wald ratio estimate of each variant, ${\widehat{\beta }}_{Y_j}/{\widehat{\beta }}_{X_j},j\in S$.

We consider limited information maximum likelihood (LIML) estimation of θ₀. The Core and Full estimators, ${\widehat{\theta }}_C$ and ${\widehat{\theta }}_F$, are given by

$${\widehat{\theta }}_C=\arg \underset{\theta }{\min }{\displaystyle \sum _{j\in S_0}\frac{{\left({\widehat{\beta }}_{Y_j}-{\widehat{\beta }}_{X_j}\theta \right)}^2}{{\sigma }_{Y_j}^2+{\sigma }_{X_j}^2{\theta }^2}\text{and}{\widehat{\theta }}_F=\arg \underset{\theta }{\min }{\displaystyle \sum _{j\in S_0\cup S}\frac{{\left({\widehat{\beta }}_{Y_j}-{\widehat{\beta }}_{X_j}\theta \right)}^2}{{\sigma }_{Y_j}^2+{\sigma }_{X_j}^2{\theta }^2}.}}$$

Under Assumptions 1-3, Theorem 3.1 of Zhao et al. (2020) shows that the asymptotic distribution of ${\widehat{\theta }}_C$ is

$${\Delta }_C^{-1/2}\left({\widehat{\theta }}_C-{\theta }_0\right)\stackrel{D}{\to }N\left(0,1\right),$$

as n, p → ∞, where ${\Delta }_C={\eta }_C^{-2}\left({\eta }_C+{\zeta }_C\right),{\eta }_C={\sum }_{j\in S_0}{\Omega }_j^{-1}{\beta }_{X_j}^2,{\zeta }_C={\sum }_{j\in S_0}{\Omega }_j^{-2}{\sigma }_{X_j}^2{\sigma }_{Y_j}^2$, and ${\Omega }_j={\sigma }_{Y_j}^2+{\theta }_0^2{\sigma }_{X_j}^2$. Using similar arguments, we can derive the asymptotic distribution of the Full estimator.

Theorem 1 (Full estimator)

Under Assumptions 1-4, and the Full estimator is consistent, and its asymptotic distribution is

$${\Delta }_F^{-1/2}\left({\widehat{\theta }}_F-{\theta }_0-b\right)\stackrel{D}{\to }N\left(0,1\right),$$

as n, p → ∞, where ${\Delta }_F={\left({\eta }_C+{\eta }_S\right)}^{-2}\left({\eta }_C+{\eta }_S+{\zeta }_C+{\zeta }_S\right),b={\left({\eta }_C+{\eta }_S\right)}^{-1}{b}_S,b_S={\sum }_{j\in S}{\Omega }_j^{-1}{\beta }_{X_j}{\tau }_j,{\eta }_S={\sum }_{j\in S}{\Omega }_j^{-1}{\beta }_{X_j}^2$, and ${\zeta }_S={\sum }_{j\in S}{\Omega }_j^{-2}{\sigma }_{X_j}^2{\sigma }_{Y_j}^2$.

The variance terms η_C and η_S are of order $O\left(n{\Vert {\beta }_X\Vert }_2^2\right)$, and the terms ς_C and ς_S are of order O(p). Therefore, (η_C + η_S)⁻¹ would be the asymptotic variance of ${\widehat{\theta }}_F$ in a fixed p setting with strong instruments, and (η_C + η_S)⁻²(ς_C + ς_S) is an additional variance component to account for the extra uncertainty due to many weak instruments (Zhao et al., 2020). This is particularly important for our instrument selection problem since under-estimated variances based on ‘fixed p’ asymptotics could cause a mean squared error-based selection criteria to falsely recommend the inclusion of additional instruments. These two variance components are of the same order of magnitude when $p/n{\Vert {\beta }_X\Vert }_2^2=\Theta \left(1\right)$.

As discussed by Newey and Windmeijer (2009), despite the knife-edge condition required to balance these variance components, it may be advisable to use the weak instrument variance correction (η_C + η_S)⁻²(ς_C + ς_S) in general scenarios when n is considerably larger than p, and when instruments are strong. For example, the simulation study of Davies et al. (2015, pp. 457-9), which mimicked an MR design, showed that standard errors that did not correct for many weak instrument effects led to inflated type I error rates even for the case where n = 3000 and p = 9.

We also note that compared with the model of Zhao et al. (2020), here we consider τ to be a fixed effect rather than a random variable; this direct variant effect on the outcome induces a bias in estimation rather than an increase in variance. The asymptotic variance of ${\widehat{\theta }}_F$ is of order $O\left(1/n{\Vert {\beta }_X\Vert }_2^2\right)+O\left(p/n^2{\Vert {\beta }_X\Vert }_2^4\right)$, and its asymptotic bias is of order $O\left(1/\sqrt{n}{\Vert {\beta }_X\Vert }_2\right)$. Thus, there is a meaningful bias-variance trade off if $p/n{\Vert {\beta }_X\Vert }_2^2=O\left(1\right)$ since the square of the asymptotic bias is of the same order of magnitude as the asymptotic variance.

To carry out focused instrument selection, we need to estimate and compare the asymptotic mean squared error (AMSE) of ${\widehat{\theta }}_C$ and ${\widehat{\theta }}_F$. The Core estimator is asymptotically unbiased, and it is straightforward to consistently estimate its asymptotic variance Δ_C. Under local mis-specification, the asymptotic bias b of the Full estimator cannot be consistently estimated. However, we can use ${\widehat{\theta }}_C$ to construct an asymptotically unbiased estimate of b.

Let ${\widehat{\eta }}_C={\sum }_{j\in S_0}{\widehat{\Omega }}_j^{-1}\left({\widehat{\beta }}_{X_j}^2-{\sigma }_{X_j}^2\right),{\widehat{\eta }}_S={\sum }_{j\in S}{\widehat{\Omega }}_j^{-1}\left({\widehat{\beta }}_{X_j}^2-{\sigma }_{X_j}^2\right)$, and ${\widehat{\Omega }}_j={\sigma }_{Y_j}^2+{\widehat{\theta }}_C^2{\sigma }_{X_j}^2$. Then, our estimator of b is $\widehat{b}={\left({\widehat{\eta }}_C+{\widehat{\eta }}_S\right)}^{-1}{\widehat{b}}_S$, where ${\widehat{b}}_S={\sum }_{j\in S}{\widehat{\Omega }}_j^{-1}{\widehat{\beta }}_{X_j}{\widehat{\beta }}_{Y_j}-{\widehat{\theta }}_C{\sum }_{j\in S}{\widehat{\Omega }}_j^{-1}\left({\widehat{\beta }}_{X_j}^2-{\sigma }_{X_j}^2\right)$.

Theorem 2 (Asymptotic bias estimator)

Under Assumptions 1-4, the asymptotic distribution of $\widehat{b}$ is

$${\Delta }_B^{-1/2}\left(\widehat{b}-b\right)\stackrel{D}{\to }N\left(0,1\right),$$

as n, p → ∞, where ${\Delta }_B={\left({\eta }_C+{\eta }_S\right)}^{-2}\left[{\eta }_S+{\zeta }_S+{\xi }_S+{\eta }_C^{-2}{\eta }_S^2\left({\eta }_C+{\zeta }_C\right)\right]$ and ${\xi }_S=2{\theta }_0^2{\sum }_{j\in S}{\Omega }_j^{-2}{\sigma }_{X_j}^4$.

The standard error of the estimate $\widehat{b}$ can be up to the same order of magnitude $O_P\left(1/\sqrt{n}{\Vert {\beta }_X\Vert }_2\right)$ as the true bias b, and hence the asymptotic variance of $\widehat{b}$ does not decrease fast enough for consistency as n, p → ∞. However, Theorem 2 shows that $\widehat{b}$ is an asymptotically unbiased estimator. Similar to before, we can write the asymptotic variance Δ_B as the sum of two terms: ${\left({\eta }_C+{\eta }_S\right)}^{-2}{\eta }_S\left(1+{\eta }_C^{-1}{\eta }_S\right)$ and ${\left({\eta }_C+{\eta }_S\right)}^{-2}\left({\zeta }_S+{\xi }_S+{\eta }_C^{-2}{\eta }_S^2{\zeta }_C\right)$. The first term is the asymptotic variance of $\widehat{b}$ in a fixed p setting with strong instruments, and therefore the second term represents the extra uncertainty in estimation due to many weak instruments.

In order to estimate the AMSE of ${\widehat{\theta }}_C$ and ${\widehat{\theta }}_F$, we need to construct consistent estimators ${\widehat{\Delta }}_C={\widehat{\eta }}_C^{-2}\left({\widehat{\eta }}_C+{\widehat{\varsigma }}_C\right)$ for ${\Delta }_C,{\widehat{\Delta }}_F={\left({\widehat{\eta }}_C+{\widehat{\eta }}_S\right)}^{-2}\left({\widehat{\eta }}_C+{\widehat{\eta }}_S+{\widehat{\varsigma }}_C+{\varsigma }_S\right)$ for Δ_F, and ${\widehat{\Delta }}_B={\left({\widehat{\eta }}_C+{\widehat{\eta }}_S\right)}^{-2}\left[{\widehat{\eta }}_S+{\widehat{\varsigma }}_S+{\widehat{\xi }}_S+{\widehat{\eta }}_C^{-2}{\widehat{\eta }}_S^2\left({\widehat{\eta }}_C+{\widehat{\varsigma }}_C\right)\right]$ for Δ_B where ${\widehat{\varsigma }}_C={\sum }_{j\in S_0}{\widehat{\Omega }}_j^{-2}{\sigma }_{X_j}^2{\sigma }_{Y_j}^2,{\widehat{\varsigma }}_S={\sum }_{j\in S}{\widehat{\Omega }}_j^{-2}{\sigma }_{X_j}^2{\sigma }_{Y_j}^2$, and ${\widehat{\xi }}_S=2{\widehat{\theta }}_C^2{\sum }_{j\in S}{\widehat{\Omega }}_j^{-2}{\sigma }_{X_j}^4$.

Since ${\widehat{\theta }}_C$ is asymptotically unbiased, a consistent estimator for its AMSE is ${\widehat{\Delta }}_C$. Whereas for ${\widehat{\theta }}_F$, an asymptotically unbiased estimator of its AMSE is $\left({\widehat{b}}^2-{\widehat{\Delta }}_B\right)+{\widehat{\Delta }}_F$. Following DiTraglia (2016), since the square of the asymptotic bias cannot be negative, we use $\max \left(0,{\widehat{b}}^2-{\widehat{\Delta }}_B\right)$ instead of ${\widehat{b}}^2-{\widehat{\Delta }}_B$ when estimating the AMSE of ${\widehat{\theta }}_F$.

Define $\widehat{W}=\max \left({\widehat{b}}^2-{\widehat{\Delta }}_B,0\right)+{\widehat{\Delta }}_F-{\widehat{\Delta }}_C$ as the estimated AMSE of ${\widehat{\theta }}_F$ minus the estimated AMSE of ${\widehat{\theta }}_C$. Therefore, the selection event that we select the Full estimator is given by $\left\{\widehat{W}\le 0\right\}$. The “Focused” estimator is then given by

$$\widehat{\theta }=I\left\{\widehat{W}\le 0\right\}{\widehat{\theta }}_F+\left(1-I\left\{\widehat{W}\le 0\right\}\right){\widehat{\theta }}_C.$$

Since both ${\widehat{\theta }}_F$ and ${\widehat{\theta }}_C$ are consistent estimators of θ₀, so is the Focused estimator $\widehat{\theta }$.

As noted by a reviewer, it is also possible to use Theorem 2 to construct a valid test of H₀ : b = 0. Such tests could potentially be used to inform an alternative instrument selection strategy based on pre-testing for bias; see Supplementary Material for further discussion (Patel et al., 2024).

4. Post-selection inference

In this section we discuss the problem of constructing confidence intervals for the Focused estimator $\widehat{\theta }$, which is non-standard because model selection is based on an AMSE criterion that is not consistently estimated.

We start by deriving the asymptotic distribution of $\widehat{\theta }$, and discuss why naive confidence intervals that ignore sampling uncertainty in instrument selection are likely to perform poorly. We then note the relative merits of two related inference procedures proposed in DiTraglia (2016), before introducing a new “Focused” approach that combines their strengths. Just as the Focused estimator aims to achieve a good balance between bias and variance, Focused intervals aim to achieve a good balance between the length of confidence intervals and the potential worst case asymptotic coverage over the likely space of direct instrument effects on the outcome.

4.1. Asymptotic distribution of the Focused estimator

A feature of the local mis-specification framework is that the sampling uncertainty from asymptotic bias estimation directly affects the asymptotic distribution of the post-selection Focused estimator $\widehat{\theta }$. In particular, there is non-ignorable uncertainty in model selection: even if the use of only the core instruments is optimal from an AMSE perspective, the uncertainty in asymptotic bias estimation can cause the Focused estimator to erroneously select the full set of instruments.

In contrast, under consistent model selection, the Focused estimator would choose either ${\widehat{\theta }}_C$ or ${\widehat{\theta }}_F$ with probability approaching 1 as n, p → ∞. While this would seem to simplify the task of inference, it would still not be possible to consistently estimate the distribution of post-selection estimators uniformly over the space of direct effects τ (Leeb and Pötscher, 2005, Section 2.3, pp. 38–40). Moreover, consistent model selection does not suit our goal of improved estimation in terms of low risk, since the worst case risk of post-selection estimators would be unbounded (Leeb and Pötscher, 2008).

Theorem 3 (Focused estimator)

Under Assumptions 1-4, the Focused estimator is consistent, and is asymptotically distributed $\widehat{\theta }\stackrel{a}{~}{\theta }_0+\Lambda \left(b\right)$ as n, p → ∞ where $\Lambda \left(b\right)=I\left\{W\left(b\right)\le 0\right\}\left(b+{\mathcal{K}}_F\right)+\left(1-I\left\{W\left(b\right)\le 0\right\}\right){\mathcal{K}}_C,W\left(b\right)=\max \left({\left(b+{\mathcal{K}}_b\right)}^2-{\Delta }_B,0\right)+{\Delta }_F-{\Delta }_C$, and

$$\mathcal{K}≔\left[\begin{array}{c}{\mathcal{K}}_F\\ {\mathcal{K}}_C\\ {\mathcal{K}}_b\end{array}\right]~N\left(\left[\begin{array}{c}0\\ 0\\ 0\end{array}\right],\Delta ≔\left[\begin{array}{c}{\Delta }_F{\Delta }_E{\Delta }_D\\ {\Delta }_E{\Delta }_C{\Delta }_A\\ {\Delta }_D{\Delta }_A{\Delta }_B\end{array}\right]\right),$$

where ${\Delta }_E={\eta }_C^{-1}{\left({\eta }_C+{\eta }_S\right)}^{-1}\left({\eta }_C+{\varsigma }_C\right),{\Delta }_A=-{\Delta }_E{\eta }_C^{-1}{\eta }_S$, and ${\Delta }_D={\eta }_C^{-1}{\left({\eta }_C+{\eta }_S\right)}^{-2}{\eta }_C\left({\eta }_S+{\varsigma }_S\right)-{\Delta }_E{\left({\eta }_C+{\eta }_S\right)}^{-1}{\eta }_S$.

Theorem 3 indicates that the asymptotic distribution of $\widehat{\theta }$ is a weighted average of the asymptotic distributions of ${\widehat{\theta }}_C$ and ${\widehat{\theta }}_F$, where the weights are random even as n, p → ∞. As a result, “naive” confidence intervals for $\widehat{\theta }$ that ignore sampling uncertainty in instrument selection should not be reported. Such intervals can perform extremely poorly in practice, with coverage arbitrarily far below their nominal level.

Under the condition $p/n{\Vert {\beta }_X\Vert }_2^2=O\left(1\right)$ as n, p → ∞ from Assumption 3, the variance components Δ can be consistently estimated, and therefore to simplify notation we henceforth assume that Δ is known.

4.2. Focused confidence intervals with coverage constraints

There are two related problems that a valid inference procedure must solve. First, confidence intervals need to be widened to account for the model selection uncertainty introduced by focused instrument selection. Second, confidence intervals need to be re-centered if the focused estimator is asymptotically biased.

If the true asymptotic bias component b was known, inference would be straightforward: Theorem 3 could be used directly to simulate the distribution of Λ(b). However, b cannot be consistently estimated in our locally invalid instruments framework. For this setting, DiTraglia (2016) proposes two feasible inference procedures that consider the distribution of Λ(b) at values of b that are plausible given the observed data.

The “1-step” interval is based on the distribution of Λ(b) evaluated at $b=\widehat{b}$, and it effectively assumes that the true value of b is equal to its asymptotically unbiased estimator $\widehat{b}$. This is intuitive since $\widehat{b}$ is in some sense the most plausible value of b given the data. It follows that if $\widehat{b}$ is close to the true value of b, then the 1-step interval will have coverage that is close to its nominal level. Moreover, when the direct effects τ are small, simulation evidence from DiTraglia (2016) suggests that the 1-step interval has competitive coverage and can be shorter in length than the “Core” interval, which is the standard confidence interval of ${\widehat{\theta }}_C$ that uses only the core instruments. More generally, however, the 1-step interval comes with no theoretical guarantees; it may substantially under-cover, although its performance is much better than that of a naive interval that ignores instrument selection uncertainty.

The reason why the 1-step interval may under-cover is that it fails to account for the uncertainty in the estimate $\widehat{b}$ of b, thus potentially leading to intervals that are too short and centered incorrectly. The “2-step” interval allows for such uncertainty by first constructing a confidence region φ for b. It then simulates the distribution of Λ(b′) at every value b′ in φ, constructing a collection of confidence intervals each based on the assumption that the true value of b is b′. To obtain a uniform coverage guarantee, the 2-step interval takes the outer envelope of all of the resulting intervals. This makes the 2-step interval extremely conservative: in general there is no value of b for which the actual coverage equals the nominal coverage, and hence it will always over-cover. Our simulation evidence suggests that this over-coverage problem makes the 2-step intervals too wide to be useful in practice.

We consider a way forward for improved inference with “Focused” intervals. These intervals aim to combine the strengths of the 1-step and 2-step intervals while avoiding their drawbacks. Like the 1-step interval, it is constructed by simulating Λ(b) at a single value of b rather than taking an outer envelope over many values. This means that it can yield shorter confidence intervals. Like the 2-step interval, however, it comes with theoretical guarantees. The key is to choose an appropriate value of b.

The Focused interval considers only values of b that are contained in a (1 – α₁) × 100% confidence interval called φ. This is the same confidence interval for b used in the 2-step interval approach. For some values of b in φ the distribution of Λ(b) will be highly dispersed. Suppose that b′ is such a value, so that a (1 – α₂) × 100% confidence interval CI(b′) for θ₀ computed under the assumption that b = b′ will be relatively wide. By construction, CI(b′) will achieve nominal coverage probability 1 – α₂ when b = b′. The key insight is as follows: if b″ is a value in φ for which Λ(b″) is relatively less dispersed, then CI(b′) may also be a nearly valid confidence interval for θ₀ when b = b″. Using this idea, the construction of the Focused interval proceeds as follows, based on a user-specified tolerance γ and nominal coverage probability 1 – α.

Algorithm 1. (Focused interval with a minimum coverage constraint).

1. Construct a (1 – α₁) × 100% confidence interval φ for b using Theorem 2.

2. For each b′ ∈ φ, calculate a collection of (1 – α₂) × 100% intervals [a_l(b′), a_u(b′)] for Λ(b′) using Theorem 3, each under the assumption that b = b′.

3. Calculate $\overline{\phi }=\left\{b^{\prime }\left(\alpha ,\gamma \right)\in \phi :P\left(a_l\left(b^{\prime }\right)\le \Lambda \left(b^{″}\right)\le a_u\left(b^{\prime }\right)\right)\ge 1-{\alpha }_2-\gamma ,\mathit{\text{for}}\mathit{\text{all}}{b}^{″}\in \phi \right\}$.

4. Find the value of $b^{\prime }\left(\alpha ,\gamma \right)\in \overline{\phi }$ that yields the shortest (1 – α₂) × 100% confidence interval for Λ(b′). Call this value b*(α, γ). A CI for θ₀ is then $\left[\widehat{\theta }-a_l\left(b*\left(\alpha ,\gamma \right)\right),\widehat{\theta }-a_u\left(b*\left(\alpha ,\gamma \right)\right)\right]$.

5. Notice that b* (α, γ) depends on γ, α₁, and α₂. Repeat steps 1-4 for a range of choices of α₁ subject to the constraint α₁ + α₂ = α. Choose the value of α₁ that yields the shortest interval for θ₀.

Like the 1-step interval, the Focused interval is always shorter than the 2-step interval because it is one of the intervals contained in the outer envelope that forms the 2-step interval. Unlike the 1-step interval, its asymptotic coverage probability can never fall below 1 – α – γ.

To illustrate the differences between the 1-step, 2-step, and Focused intervals, Figure 2 plots the distributions of $\Lambda \left(\overline{b}\right)$ evaluated at different bias levels $\overline{b}$. The true bias is b, an asymptotically unbiased estimator for the bias is $\widehat{b}$, while b*(α, γ) and b″ are other bias levels contained in a (1 – α₁) × 100% confidence region φ for b. Since b is unknown, we cannot directly calculate a confidence interval based on the distribution of Λ(b) given in green. The 1-step (1 – α) × 100% confidence interval is based on the distribution of $\Lambda \left(\widehat{b}\right)$ given in blue. The 2-step interval takes an outer envelope of (1 – α₂) × 100% confidence intervals based on the distributions of $\Lambda \left(\widehat{b}\right)$ in blue, Λ(b*(α, γ)) in yellow, Λ(b″) in red, and all other distributions of $\Lambda \left(\overline{b}\right)$ for every $\overline{b}$ in φ.

Fig 2.

The distribution of $\Lambda \left(\overline{b}\right)$ evaluated at $\overline{b}=b,\overline{b}=\widehat{b}$, and $\overline{b}=b*\left(\alpha ,\gamma \right)$, for the case where all instruments are equally strong, and the additional instruments are invalid.

The Focused interval is a (1 – α₂) × 100% confidence interval based on the yellow distribution of Λ(b*(α, γ)) in Figure 2, and this interval also covers (1 – α₂ – γ) × 100% of the distributions of $\Lambda \left(\widehat{b}\right)$ and Λ(b″), and all other distributions $\Lambda \left(\overline{b}\right)$ for every $\overline{b}$ in φ. Note that the distribution of Λ(b*(α, γ)) that the Focused interval is based on is changing with the choice of γ. For lower values of γ, Λ(b*(α, γ)) is relatively dispersed: a bias level b*(α, γ) that corresponds to a more dispersed distribution of Λ(b*(α, γ)) is selected in order to satisfy the more stringent (1 – α₂ – γ) × 100% coverage requirement of the distributions of $\Lambda \left(\overline{b}\right)$, for all $\overline{b}\in \phi$.

Theorem 4 (Worst case asymptotic coverage of Focused intervals)

Under Assumptions 1-4, the Focused interval defined in Algorithm 1 has asymptotic coverage probability no less than 1 – α – γ as n, p → ∞.

The Focused interval is designed to achieve nominal coverage 1 ‒ α at a plausible value of the asymptotic bias, while also controlling the worst case asymptotic coverage according to a maximum allowable size distortion γ. The choice of γ dictates the trade off between the worst case coverage level and the length of the interval, with a lower level of tolerance γ more likely to provide conservative inference. The feasibility of constructing the Focused interval relies on the existence of a sufficiently dispersed distribution of Λ(b′(α,γ)) at a plausible value b′(α,γ), which may not exist for extremely low levels of γ. However, selecting an extremely low level of γ would defeat the purpose of the Focused interval, which aims to be competitive in terms of both coverage and length.

Although Algorithm 1 is novel, the concept of an inference procedure that depends on a user-specified allowable size distortion has precursors in the econometrics literature. For example, Andrews (2018) proposes an inference strategy that controls a worst case coverage distortion under weak instruments, and where the decision to report a conventional or weak instrument robust confidence set depends on the level of under-coverage that an investigator is willing to accept.

5. Simulation study

In this section we illustrate how the finite sample performance of the Focused estimator and Focused confidence interval depends on the strength of instruments and the magnitude of direct variant effects on the outcome.

First, we consider estimation performance in terms of root-mean squared error (RMSE). Second, we show how the Focused interval may be able to achieve a favourable balance of length and coverage, and discuss where it may lead to improved inference compared with the Core interval, which is the conventional confidence interval of ${\widehat{\theta }}_C$ based on using only the core instruments. Third, we discuss the sensitivity of the Focused interval to the choice of γ which controls the worst case coverage loss. Finally, we consider the performance of the Focused estimator when the core instruments S₀ are in fact invalid.

5.1. Design

We simulated two-sample summary data on p = 110 variants according to Assumptions 1–4. The sample sizes were set at n = n_X = n_Y = 1000. Of the 110 variants, 10 were set to be valid instruments, and they formed the core set S₀. The remaining p – |S₀| = 100 variants formed the set of additional instruments S.

We generated estimated associations ${\widehat{\beta }}_{X_j}~N\left({\beta }_{X_j},{\sigma }_{X_j}^2\right)$ and ${\widehat{\beta }}_{Y_j}~N\left({\beta }_{Y_j},{\sigma }_{Y_j}^2\right)$, where true genetic variant associations with the risk factor were set as ${\beta }_{X_j}={\overline{\beta }}_C/\sqrt{\left|S_0\right|}$ for j ∈ S₀, and ${\beta }_{X_j}={\overline{\beta }}_S/\sqrt{p-\left|S_0\right|}$ for j ∈ S, where ${\overline{\beta }}_C$ and ${\overline{\beta }}_S$ were chosen to maintain a particular level of the concentration parameters ${\lambda }_C={\displaystyle {\sum }_{j\in S_0}}{\beta }_{X_j}^2/\left(\left|S_0\right|{\sigma }_{X_j}^2\right)$ and ${\lambda }_S={\sum }_{j\in S}{\beta }_{X_j}^2/\left(\left|S\right|{\sigma }_{X_j}^2\right)$, which are measures of the average instrument strength of S₀ and S. The variances ${\sigma }_{X_j}^2$ and ${\sigma }_{Y_j}^2$ were set equal to 1/n for all variants.

The true variant–outcome associations were set to be β_Yj = β_Xj θ₀ for j ∈ S₀, and β_Yj = β_Xj θ₀ + τ_j for j ∈ S, where the true causal effect was θ₀ = 0.2, and the direct effects are fixed effects generated as ${\tau }_j~U\left[0,\overline{\tau }/\sqrt{np}\right]$, for different values $\overline{\tau }\ge 0$.

For inference, we set the nominal coverage probability at 1 – α = 0.95, and unless otherwise stated, the allowable worst case size distortion for the Focused interval was set at γ = 0.2. Along with the Focused, 1-step, 2-step, and Core intervals described above, we also note the performance of the “Naive” confidence interval, which is the standard confidence interval for the selected estimator that ignores sampling uncertainty in model selection.

5.2. Estimation

Figure 3 highlights that when the direct variant effects on the outcome are sufficiently small, the RMSE of the Focused estimator is lower than the Core estimator. However, this improvement is not uniform across larger values of $\overline{\tau }$. The performance of the Focused estimator worsens for more intermediate values of $\overline{\tau }$. Then, as $\overline{\tau }$ becomes large, the Focused estimator should select the Core estimator with large probability, so that the RMSEs of the Core and Focused estimators should be quite similar.

Fig 3.

RMSE of Focused estimator relative to RMSE of Core estimator varying with the average instrument strength of S₀ (λ_C) and S (λ_S), and invalidness of $S\left(\overline{\tau }\right)$.

The extent of the improvement in RMSE that is possible appears to depend on the strength of instruments. For example, when $\overline{\tau }=2$, the Focused estimator offered a 32.8% reduction in RMSE when all instruments are relatively weak (λ_C = λ_S = 40), a 30.7% reduction when λ_C = λ_S = 120, and a 27.8% reduction when all instruments are relatively strong (λ_C = λ_S = 200).

The relative strengths of the core and additional instrument sets affect the values of $\overline{\tau }$ over which focused instrument selection is able to improve estimation. When the additional instruments were strong and the core instruments were relatively weak (λ_C = 40, λ_S = 200), the Focused estimator had a lower RMSE than the Core estimator over the range $0\le \overline{\tau }<10$. In contrast, when λ_C = 200 and λ_S = 40, the Focused estimator had a lower RMSE only over the range $0\le \overline{\tau }\le 2$.

In summary, these estimation results intuitively suggest that focused instrument selection is more likely to improve estimation when the additional instruments are not too invalid, the core instruments are quite weak, and the additional instruments are strong. This is practically relevant for MR analyses of polygenic traits where several genes may be causally related.

5.3. Confidence intervals

For inference, Figure 4 shows that the coverage probability of the Naive interval dropped to as low as 0.4 when all instruments are equally strong $\left(\overline{\tau }=8\right)$, and lower than 0.2 when the additional instruments are much stronger than the core instruments (λ_C = 40, λ_S = 200, $\overline{\tau }=10$). This underscores the importance for confidence intervals of $\widehat{\theta }$ to account for sampling uncertainty in model selection. On the other hand, the 2-step intervals are conservative; the intervals exceeded nominal coverage probability, and Figure 5 shows that they were generally over 30% longer in length than the Core intervals.

Fig 4. Coverage probabilities of confidence intervals (nominal coverage is 1 – α = 0.95).

Fig 5. Length of confidence intervals relative to the Core interval (nominal coverage is 1 – α = 0.95).

The 1-step intervals are a useful compromise between the 2-step and Naive intervals. When the additional instruments were not too invalid (i.e. for small enough parameter values of $\overline{\tau }$), the 1-step intervals were shorter in length than the Core intervals, while also achieving nominal coverage probability. The performance of the 1-step interval appears to be quite sensitive to the relative strengths of the core and additional instruments. When the additional instruments were relatively strong (for example, λ_C = 40, λ_S ≥ 160), the 1-step intervals were shorter than the Core intervals, but they under-covered for large enough values of $\overline{\tau }$. Conversely, when the core instruments are strong enough, the 1-step intervals showed no real advantages compared with the Core interval.

For the Focused intervals, we selected the allowable worst case size distortion as γ = 0.2, so that for a 1 – α = 0.95 level confidence interval, coverage probability should not be lower than 0.75. Figure 4 shows that the coverage of the Focused interval was higher than 0.8 for all parameter values $\overline{\tau }$, whereas for some values the coverage probability of the 1-step interval dropped below 0.7. Moreover, the coverage of the Focused interval was generally more competitive than the 1-step interval, especially when the additional instruments were at least as strong as the core instruments.

When the core instruments were not much weaker than the additional instruments, the Focused intervals were shorter in length than the 1-step interval. Interestingly, the Focused intervals were also up to 8% shorter in length than the Core intervals unless the additional instruments were very invalid $\left(\overline{\tau }\ge 12\right)$. The Focused interval can therefore improve the power of an analysis by incorporating information from additional relevant instruments if they are not too invalid. At the same time, the Focused interval also retains good size control and buys insurance against serious under-coverage when additional instruments are very invalid.

5.4. Sensitivity to the choice of γ

The Focused intervals require investigators to choose an acceptable level of the worst case size distortion γ. Figure 6 illustrates how the performance of the Focused interval varies according to the choice of γ for the case where all instruments are equally strong (λ_C = λ_S = 40). The results of other confidence intervals discussed in this paper are also shown for comparison, but of course their performance should not vary with γ.

Fig 6.

The dashed line in the first row is the allowable size distortion 1 – α – γ (nominal coverage is 1 – α = 0.95). The second row plots the length of confidence intervals relative to the Core interval.

Figure 6 verifies that the Focused interval is able to control the worst case coverage over all parameter values $\overline{\tau }$. The cost of allowing only a small size distortion is the longer length of the interval. The Focused intervals were over 15% longer in length than the Core intervals when γ = 0.05, although they were also much shorter than the 2-step intervals.

Conversely, for larger values of γ, the Focused intervals become shorter, but only to a certain degree: since the Focused intervals account for uncertainty in model selection, the lengths of the intervals are not as short as the Naive intervals. Accordingly, the coverage probability of the Focused intervals will also not change for large enough values of γ.

5.5. Estimation when S₀ contains invalid instruments

Focused instrument selection aims to prioritise the evidence suggested by a core set of genetic variants S₀ that are believed to be valid instruments. In practice, investigators may not always be correct in their belief, and S₀ may actually consist of invalid instruments. We consider this setting in simulation by slightly altering the design in Section 5.1 so that the true variant–outcome associations are set to be β_Yj = β_Xjθ₀ + τ_j, where ${\tau }_j~U\left[0,{\overline{\tau }}_C/\sqrt{np}\right]$ for j ∈ S₀. Figure 7 presents the estimation results for the case where all instruments are equally strong (λ_C = λ_S = 40).

Fig 7.

RMSE of Focused estimator relative to RMSE of Core estimator varying with the invalidness of $S_0\left({\overline{\tau }}_C\right)$ and invalidness of $S\left(\overline{\tau }\right)$.

Our results show that when S₀ contained only slightly invalid instruments, the Focused estimator was able to improve on the Core estimator in terms of RMSE. As the instruments in S₀ become more invalid, but not as invalid as S, the performance of the Focused estimator worsens because the estimated bias from including S is under-estimated. We note that the Focused estimator also performed relatively well when the instruments S₀ were at least as invalid than S; for example, the RMSE of the Focused estimator was lower than that of the Core estimator when ${\overline{\tau }}_C=\overline{\tau }=12$.

6. Empirical Examples

In this section, we demonstrate how focused instrument selection can be applied in MR studies. First, we consider the problem of instrument selection in drug target MR, where variation in a gene that encodes the protein target of a drug is used to proxy drug target perturbation. Such MR investigations have the potential to provide genetic evidence on drug efficacy and to study potential side effects (Gill et al., 2021).

An important decision in drug target MR is to specify a “cis window” that defines a gene region from which instruments are selected. In practice, investigators often use cis windows that are wider than gene regions defined by tools such as GeneCards (Stelzer et al., 2016) in order to boost the power of an analysis through the use of multiple instruments. This leaves the possibility of a type of publication bias where researchers may report findings only from a cis window that gives their preferred result. In Section 6.1, we apply the focused instrument selection method to two lipid drug targets, where genetic variants that may be included only from widening a cis window are additional instruments.

Second, we investigate the effect of vitamin D supplementation on a range of outcomes. GWASs have identified strong genetic associations in biologically plausible genes known to have a functional role in the transport, synthesis, or metabolism of vitamin D. Some previous MR studies aiming to study vitamin D effects have used genetic variants located in neighborhoods of those genes to instrument vitamin D as they are considered more likely to satisfy the exclusion restriction than other genome-wide significant variants (Mokry et al., 2015; Revez et al., 2020). However, the role of many other genes which are robustly associated with vitamin D may not yet be fully understood; for example, Jiang, Ge and Chen (2021) selected genetic variants from 69 independent loci to instrument vitamin D. In Section 6.2, we use genetic variants from biologically plausible genes as the core set of instruments, and apply focused instrument selection to select from many additional genetic variants which may be considered more likely to have a direct effect on the outcome through their effects on traits other than vitamin D.

For Focused confidence intervals, we selected γ = 0.2 for the maximum allowable size distortion. The data used for our analyses are publicly available through the MR-Base platform (Hemani et al., 2018), and R code to perform our empirical investigation is available on GitHub at github.com/ash-res/focused-MR/.

6.1. CETP and PCSK9 inhibitors

Cholesteryl ester transfer protein (CETP) inhibitors are a class of drugs that raise high density lipoprotein cholesterol levels and lower low density lipoprotein cholesterol (LDL-C) levels. At least three CETP inhibitors have failed in clinical trials to conclude a protective effect against coronary heart disease (CHD), but the successful trial of Anacetrapib showed a modest benefit when used with statins (Bowman et al., 2017). A recent drug target MR analysis by Schmidt et al. (2021) offers genetic evidence that CETP inhibition may be an effective approach for preventing CHD. Here we investigate the robustness of a similar drug target MR study to the choice of a cis window used to select instruments.

We study the genetically predicted LDL-C lowering effect of CETP inhibition on a range of outcomes by using genetic variants located in a neighborhood of the CETP gene. We may consider instruments drawn from the “narrow” cis window Chr 16: bp: 56,985,862–57,027,757 (which is the region stated in GeneCards ±10,000 bp) to more accurately represent the genetically predicted effects of CETP inhibition. At the same time, it is also common for drug target MR studies to use a “wider” cis window for instrument selection. We may consider potential additional instruments from the wider window Chr 16: bp: 56,895,862–57,117,757 (which is the region stated in GeneCards ±100,000 bp). The use of these additional instruments may be considered more likely to lead to biased estimation compared with using only those variants from the narrow cis window.

Therefore the Core estimator used only the 3 uncorrelated genetic variants located in the narrow window as instruments, while the number of additional variants that were available varied between 4 and 10 depending on the outcome of interest. The Focused estimator selected the full set of instruments for 12 out of the 16 outcomes we studied. The concentration parameter for the core instruments was 91.10, while the concentration parameter for the additional instruments was between 6.52 and 8.12 depending on how many additional instruments were available for the application.

From Figure 8, we find that the genetically predicted LDL-C lowering effect of CETP inhibition is associated with a lower risk of CHD. The results also suggest that genetically predicted CETP inhibition may have a protective effect on other cardiovascular disease outcomes; specifically atrial fibrillation and heart failure. We do not find evidence for a protective effect on stroke outcomes, nor do we find evidence for any adverse effects on various non-cardiovascular related outcomes.

Fig 8.

CETP gene analysis. Point estimates and 95% confidence intervals of the change in log odds ratio of various outcomes due to a 1 standard deviation increase in instrumented LDL-C.

Compared with CETP inhibitors, PCSK9 inhibitors (PCSK9i) are a more established class of drugs that lower LDL-C levels. A drug target MR analysis can genetically proxy the effect of taking PCSK9i by instrumenting LDL-C using genetic variants located in a neighborhood of the PCSK9 gene. Similar to the CETP gene analysis above, we consider variants located in a narrow cis window of PCSK9 (Chr 1: bp: 55,505,221–55,530,525; exactly equal to the window stated in GeneCards) as core instruments, while additional variants taken from a wider window ±100,000 bp are additional instruments.

Using this criteria, there were 4 core instruments when studying the same outcomes as before, apart from the case of lung cancer where there were 3. The number of additional instruments available ranged from 2 to 13. The Focused estimator again selected the full set of instruments for 12 out of 16 outcomes. The set of core instruments was very strong compared with the additional instruments; the concentration parameter for the core instruments was 618.71, compared with a range of 8.72 to 26.63 for the full set of additional variants available.

The results in Figure 9 suggest that the genetically predicted LDL-C lowering effect of PCSK9i is associated with a lower risk of coronary artery disease, heart failure, and stroke incidence; in addition, the Focused intervals suggest an association specifically with large artery stroke incidence. We also find genetic evidence that PCSK9 inhibition may adversely affect the risk of developing Alzheimer’s disease, which supports the findings of Williams et al. (2020) and Schmidt et al. (2021).

Fig 9.

PCSK9 gene analysis. Point estimates and 95% confidence intervals of the change in log odds ratio of various outcomes due to a 1 standard deviation increase in instrumented LDL-C.

6.2. Vitamin D supplementation

Finally, we apply focused instrument selection to estimate the genetically predicted effect of vitamin D supplementation on a range of outcomes. Previous MR studies instrumenting vitamin D have used variants from genes implicated in the modulation of 25OHD levels through known mechanisms. In particular, the GC, DHCR7, CYP2R1 and CYP24A1 genes have known functions in vitamin D transport, synthesis, or metabolism (Berry et al., 2012; Mokry et al., 2015). Therefore we take genetic variants from neighborhoods of these genes (±500,000 bp on regions stated in GeneCards) as our set of core instruments.

Moreover, GWASs also provide data on many other robustly associated genetic variants with vitamin D (Jiang, Ge and Chen, 2021). We use other variants passing a genome-wide significance threshold (p-value association with vitamin D less than 5 × 10⁻⁸) to form additional instrument sets. Instead of choosing between the Core and Full estimator, in our analysis of vitamin D effects we allowed the Focused estimator to also choose from subsets of the additional instruments. We partitioned the additional instruments into 3 groups by k-means clustering based on the ratio estimate of each variant $\left({\widehat{\beta }}_{Y_j}/{\widehat{\beta }}_{X_j},j\in S\right)$ and then considered all possible combinations of these 3 partitions, thus creating 7 sets of additional instruments.

In nearly all the outcomes we considered, there were between 10 and 11 genome-wide significant variants from the GC, DHCR7, CYP2R1 and CYP24A1 gene regions which were used as core instruments for vitamin D. For the outcomes primary biliary cirrhosis and asthma there were only 4 variants available to use as core instruments. Many additional instruments (≥ 50) were selected by the Focused estimator in all cases. The core instruments were again stronger than the additional instruments; for the core instruments the concentration parameter ranged from 625.52 to 723.21, and for the additional instruments it ranged from 49.73 to 99.80.

Evidence from observational studies suggests that low serum vitamin D levels are associated with an increased risk of cardiovascular disease (Dobnig et al., 2008). These reported associations may be due to unmeasured confounding, as evidence from a meta-analysis of 21 randomized clinical trials suggests no causal link (Barbarawi et al., 2019). From Figure 10, we find no genetically predicted effect of vitamin D on cardiovascular outcomes.

Fig 10.

Vitamin D effects. Point estimates and 95% confidence intervals of the change in log odds ratio of various outcomes due to a 1 standard deviation increase in instrumented vitamin D levels.

Our analysis is able to highlight that using the full set of available instruments can sometimes lead to very different estimates than when the core instruments are prioritised. In particular, the standard confidence intervals of the Full estimator suggest a non-null association of vitamin D on coronary artery disease, heart failure, eczema, primary biliary cirrhosis, and type 2 diabetes, but these results are not supported by the Focused intervals.

Our results suggest that higher vitamin D level may have a protective effect on the incidence of multiple sclerosis, a finding which has previously been discussed in other MR studies (Mokry et al., 2015). Through the Core and Focused intervals, we also find that genetically predicted vitamin D level may be associated with anorexia.

The 1-step and Focused intervals of the Focused estimator suggest that higher vitamin D level may have a protective effect on the risk of developing Alzheimer’s disease, which supports the findings from Jiang, Ge and Chen (2021), but interestingly this is not supported by the Core and Full intervals. The Core estimator used 11 instruments, the Full estimator used 91 additional instruments, and the Focused estimator selected a subset of 65 of the additional instruments. This illustrates the ability of the focused instrument selection method to carefully select additional instruments that can potentially improve the power of an analysis.

7. Conclusion

Publicly available GWAS summary data have revealed that hundreds of genetic variants are robustly associated with a wide range of traits and diseases. However, MR studies in practice do not always make use of all genetic variants that are strongly associated with risk factors of interest; in some applications, investigators may have greater confidence in the instrument validity of only a smaller subset of many genetic variants. For this setting, we propose a way forward for improved estimation through focused use of many weak and potentially invalid instruments.

Whether focused use of invalid instruments can in turn improve inference is an open question. While a uniform improvement is not possible, we propose a new strategy for postselection inference through Focused intervals, which are shown to achieve a good balance between precision and coverage probability while also guarding against a user-specified worst case size distortion. Our empirical applications highlight the potential of focused instrument selection to uncover new causal relationships in MR studies.

For neither estimation nor inference do we obtain uniform improvements, but such results could be explored under different settings. For multivariable Mendelian randomization where we are interested in estimating the effects θ₀ = (θ₁,…, θ_K)′ that multiple exposures may have on an outcome, it would be interesting to consider uniformly improved estimation in terms of an aggregate risk function, in similar spirit to Rosenman et al. (2023). Compared with the usual frequentist notion of inference, uniform improvement in terms of both length and coverage is possible from the perspective of empirical Bayes (Casella and Hwang, 2012, p.56) or “average coverage” (Armstrong, Kolesár and Plagborg-Møller, 2022) criterions. Future work could explore empirical Bayes confidence intervals under quite general restrictions on the direct effects τ_j, j ∉ S₀.

Supplementary Material

Supplementary Material A

This contains proofs of theoretical results, and further simulation results.

Supplementary Material B

This contains R code to apply the focused instrument selection method in two-sample Mendelian randomization studies.