Selection bias when inferring the effect direction in Mendelian randomization

Dear Editor,

In our recent paper (Lutz, Wu, et al., 2021), we examined the ability of the MR Steiger approach by Hemani et al. (2017) to infer the effect direction under pleiotropy, measurement error, and unmeasured confounding. In their letter to the editor, Hemani et al focus on two aspects of our paper: the data analysis and the role of unmeasured confounding. Hemani et al also describe additional problems/issues with the original MR Steiger method (Hemani et al., 2017) that we did not discuss in our original paper.

In our paper (Lutz, Wu, et al., 2021), the purpose of our data analysis was to provide an example of when the MR Steiger approach can provide obviously incorrect conclusions. We used the MR Steiger approach to examine the role of current smoking status (current vs. former smoker) on forced expiratory volume in 1 s (FEV₁) in the COPDGene study, a case-control study of chronic obstructive pulmonary disease (COPD) in current and former smokers. In our analysis, the MR Steiger approach concluded that FEV₁ causes current smoking status. We attributed this result to possible pleiotropy between the chromosome 15q25 region with smoking behavior and lung function. We agree with Hemani et al that this result could also have been due to selection bias, measurement error, unmeasured confounder, or other reasons that can also bias the MR Steiger approach. In addition, Hemani et al. raise the very important point that the MR Steiger approach is susceptible to selection bias and can provide spurious results in ascertained studies, possibly severely limiting its application to case/control studies.

Since the COPDGene study recruited COPD cases and controls who smoked at least 10 pack-years of cigarettes, Hemani et al. reperformed the analysis in the UK Biobank. They consider FEV₁ in the UK Biobank (n = 421,986) and four smoking phenotypes: ever versus never-smoking status (n = 461,066) in the UK Biobank, ever stopped smoking for 6+ months (n = 113,230) in the UK Biobank, cigarettes per day from a meta-analysis (Liu et al., 2019) (n = 337,334), and current vs former smoking status from a meta-analysis by Furberg et al. (Tobacco and Genetics Consortium, 2010) (n = 41,969). We do not understand why Hermani et al. did not consider current versus former smoking status in the UK Biobank for their reanalysis, although it is available in the UK Biobank and is the phenotype that we used in our original analysis. For the separate SNP analysis, Hemani et al only considered current vs former smoking status. However, rather than using the unascertained UK Biobank data for this analysis, Hermani et al. used the meta-analysis by Furberg et al. (Tobacco and Genetics Consortium, 2010), which is based on multiple studies including case/control studies. Unless the goal was to evaluate selection bias, the more appropriate data for this analysis would have been the UK Biobank.

For our reanalysis in the UK Biobank, we used the code provided by Hemani et al. to perform the separate SNP analysis for the other three smoking phenotypes and the UK Biobank’s current versus former smoking status. All the code we used to perform this analysis is included in the supplement. As seen in Table 1 for both smoking phenotypes (ever versus never-smoking status and current versus former smoking status in the UK Biobank), there is a significant p-value for the Steiger correlation, and the incorrect direction is detected for some of the SNPs, that is, FEV₁ causes smoking status. This differs from the results by Hemani et al., which used current versus former smoking status from a meta-analysis and not the UK Biobank.

TABLE 1.

We considered the single SNP analysis for FEV₁ from the UK Biobank and the following smoking phenotypes

		Ever/Never smoker (UK Biobank)		Quit smoking 6+ months (UK Biobank)		Cigarettes per day (Lui et al., 2019)		Former/current (UK Biobank)
SNP	Chr	Correct Steiger	Steiger p value	Correct Steiger	Steiger p value	Correct Steiger	Steiger p value	Cor (G,X)*	Cor (G,Y)	Correct Steiger	Steiger p value
rs56113850	19	FALSE	0.10	FALSE	0.72	TRUE	1.2e–33	−0.024	−0.01	TRUE	9e–5
rs7260329	19	FALSE	0.40	FALSE	0.82	TRUE	4.9e–6	0.009	0.01	FALSE	0.80
rs11858836	15	FALSE	0.02	TRUE	0.94	TRUE	1.5e–64	0.004	−0.02	FALSE	2e–11
rs72738786	15	FALSE	8.0e–3	FALSE	0.99	TRUE	2.7e–82	0.004	−0.02	FALSE	2e–11
rs11633958	15	FALSE	0.02	TRUE	0.91	TRUE	1.4e–81	0.005	−0.02	FALSE	3e–12
rs8192482	15	FALSE	0.03	TRUE	0.86	TRUE	1.7e–82	0.005	−0.02	FALSE	3e–12
rs2869548	15	FALSE	0.04	TRUE	0.77	TRUE	3.6e–73	0.004	−0.02	FALSE	2e–11

Note: The “Correct Steiger” column is TRUE if smoking causes FEV₁ and FALSE otherwise. Results in bold indicate that the correct Steiger direction was FALSE and the p < 0.05. Note that Cor(G,X)*, the correlation between the SNP G and former/current smoking status in the UK Biobank, is calculated using the recommended get_r_from_lor function from the TwoSampleMR package since the exposure is binary. Note that the SNPs are in LD and caution should be used for the overall test.

Regarding Hemani’s criticism about our choice for a single SNP analysis, we note that the original paper established the MR Steiger method (Hemani et al., 2017) and its features (simulation studies) for the single SNP analysis. However, Hemani et al. (2017) suggest that the MR Steiger directionality test should be performed for multiple SNPs. The MR Steiger (Hemani et al., 2017) paper states: “We also note that it is straightforward to extend the MR Steiger approach to multiple instruments, requiring only that the total variance explained by all instruments be calculated under the assumption that they are independent.” Nevertheless, the extension to multiple SNPs is not provided in the paper. In the TwoSampleMR R-package, Hemani et al implement a multiple SNP approach by calculating the correlation between the phenotype (i.e., exposure or outcome) and the SNPs as the square root of the sum of the correlations for each SNP and phenotype pair. This approach to combine the correlations is not explicitly discussed at all in the original paper and it is not clear why this approach is even valid, as the proposed combining of correlations is not range consistent. For that reason, we considered the SNPs separately in our analysis.

Regarding their point about unmeasured confounding, we agree with Hemani et al. that the MR Steiger method can be biased in the presence of unmeasured confounding. This is an error on our side. Our DAG reasoning did not alert us to this subtle point: it shows that unmeasured confounding does not bias the correlation when it is zero, but is silent as to how a nonzero correlation may be affected. Unmeasured confounding indeed affects phenotypic variability, which in turn may distort comparisons of the magnitude of correlations, as made by the MR Steiger approach (Lutz, Voorhies, et al., 2021).

DATA AVAILABILITY STATEMENT

Data sharing is not applicable to this article as no new data were created or analyzed in this study.