Figure 3. True discovery rate and informative covariates.

a) Schematic representation of the density f_i, which is composed of the alternative density f_1,i weighted by its prior probability π_1,i and the uniform null density weighted by π_0,i. b-d) The true discovery rate (tdr) of individual tests can vary. In b), the test has high power, and π_0,i is well below 1. In c), the test has equal power, but π_0,i is higher, leading to a reduced tdr. In d), π_0,i is like in b), but the test has little power, again leading to a reduced tdr. e) If an informative covariate is associated with each test, the distribution of the p-values from multiple tests is different for different values of the covariate. The contours represent the joint density of p-values and covariate. The BH procedure accounts only for the p-values and not the covariates (dashed red line). In contrast, the decision boundary of IHW is a step function; each step corresponds to one group, i. e., to one weight. f) By Equation (1), the density of the tdr also depends on the covariate. The decision boundary of the BH procedure (dashed red line) leads to a suboptimal set of discoveries, in this example with higher than optimal tdr for intermediate covariate values and too low otherwise. In contrast, IHW approximates a line of constant tdr, implying efficient use of the FDR budget. An important feature of IHW is that it works directly on p-values and covariates rather than explicitly estimating the tdr.