Single variant and multi-variant trend tests for genetic association with next generation sequencing that are robust to sequencing error

Abstract

As with any new technology, next generation sequencing (NGS) has potential advantages and potential challenges. One advantage is the identification of multiple causal variants for disease that might otherwise be missed by SNP-chip technology. One potential challenge is misclassification error (as with any emerging technology) and the issue of power loss due to multiple testing.

Here, we develop an extension of the linear trend test for association that incorporates differential misclassification error and may be applied to any number of SNPs. We call the statistic the linear trend test allowing for error, applied to NGS, or LTT_ae,NGS. This statistic allows for differential misclassification.

The observed data are phenotypes for unrelated cases and controls, coverage, and the number of putative causal variants for every individual at all SNPs. We simulate data considering multiple factors (disease mode of inheritance, genotype relative risk, causal variant frequency, sequence error rate in cases, sequence error rate in controls, number of loci, and others) and evaluate type I error rate and power for each vector of factor settings. We compare our results with two recently published NGS statistics. Also, we create a fictitious disease model, based on downloaded 1000 Genomes data for 5 SNPs and 388 individuals, and apply our statistic to that data.

We find that the LTT_ae,NGS maintains the correct type I error rate in all simulations (differential and non-differential error), while the other statistics show large inflation in type I error for lower coverage. Power for all three methods is approximately the same for all three statistics in the presence of non-differential error.

Application of our statistic to the 1000 Genomes data suggests that, for the data downloaded, there is a 1.5% sequence misclassification rate over all SNPs.

Finally, application of the multi-variant form of LTT_ae,NGS shows high power for a number of simulation settings, although it can have lower power than the corresponding single variant simulation results, most probably due to our specification of multi-variant SNP correlation values.

In conclusion, our LTT_ae,NGS addresses two key challenges with NGS disease studies; first, it allows for differential misclassification when computing the statistic; and second, it addresses the multiple-testing issue in that there is a multi-variant form of the statistic that has only one degree of freedom, and provides a single p-value, no matter how many loci.

Introduction

By now, it is well established that causal variants (CVs) will play an important role in disease gene mapping for complex traits. By CV, we mean an allele at a polymorphic locus that increases risk of acquiring a disease. Over the course of the last several decades, CVs have unquestionably played an important role in Mendelian diseases; that is, diseases for which the probability of acquiring the disease conditional on having at least one copy of CV is large (near one hundred percent) [1].

One of the critical issues when applying statistical association methods with CVs is that of CV frequency (CVF). Mathematically, if a SNP locus has two variants, a (non-causal), and A (causal), the CVF = Pr(A) in the population. As computed using methods implemented the programs Genetic power Calculator [2], CaTS [3], and others, statistical power to detect association with a single CV when its frequency is very small (< 0.01) and the genotype relative risks are moderate (≤2.0) is very low, for any fixed significance level and any “reasonable” sample (say, 1000 cases, 1000 controls). Equivalently, to achieve a statistical power of 80% for any fixed significance level requires an extremely large number of cases and controls (> 10,000 each). For example, using the PAWE-PH website [4], if the genotype relative risk of the heterozygote is 1.5, the disease mode of inheritance is multiplicative, the disease allele frequency is 0.01, and the SNP marker locus is the disease locus, then we require 13,926 cases and 13,926 controls to achieve 80% power at the 10E-4 significance level, even without phenotype misclassification.

Furthermore, the issue of misclassification of the CV call becomes extremely important. Several researchers have investigated the effects of genotype misclassification on the power and robustness of statistical association methods [5–23] As documented by Kang et al. [24,25] and Ahn et al. [26,27], and as can be illustrated using the method implemented in the PAWE software [28] among others, the limit of the “cost” of genotype misclassification approaches ∞ as the CV frequency approaches 0. Here, cost is defined as the ratio:

(Minimum sample size necessary to detect association in presence of misclassification):(Minimum sample size necessary to detect association when no misclassification is present).

In other words, when the CVF is rare, even very small misclassification rates (say 1%) require prohibitively large sample sizes to detect association.

A number of statistical approaches have been proposed to deal with this situation [10,11,16,29–36]. A brief summary of findings is that there are two types of misclassification errors with which to be concerned; non-differential misclassification, in which error penetrances at a locus are the same in the affected and unaffected samples; and differential misclassification, in which penetrances differ among affected and unaffected samples. The former affects only statistical power in case-control association studies, but affects both power and type I error rate in family-based association. The latter affects type I error rates and statistical power even in case-control association studies.

A recent publication has looked at non-differential misclassification in NGS studies [37]. These authors found that even at very low error rates, misclassifying a common homozygote as a heterozygote causes (sometimes substantial) loss of statistic power to detect association, and this power loss grows as the minor allele frequency decreases. These results are consistent with the work of Kang et al. [24,25], who observed the same consequences for genotype misclassification.

From our perspective, differential misclassification error is more troublesome statistically, since one no longer knows the true null distribution and therefore, true power cannot be determined.

One of the primary approaches for studying CVs is direct sequencing of the individuals of interest. In the past several years, the development of the next generation sequencing (NGS) technology has made whole-exome (the protein coding regions of a genome) or whole-genome sequencing an increasingly important component of medical genetic studies [38–49]. In addition to identifying disease-causing mutations (variants) in classic Mendelian inheritance disorders, the whole-exome/genome sequencing data has been used to identify CVs in complex diseases and “orphan” diseases that occur in a very small number of individuals (e.g., [50]). The whole-genome sequencing approach is particularly powerful when heterogeneity is present in the disease. For example, in a study of Kabuki syndrome, 32 different mutations were found in the MLL2 gene among the 53 families, including 12 de novo mutations [51]. Under this setting, the exceedingly low allele frequency of individual causal mutations makes misclassification a major problem.

The previous paragraph documents the clinical importance of identifying very rare variants. The focus of our current work is an assessment of a new test statistic applied to more common variants (e.g., where the CVF is at least 1%). We provide more information in the Discussion.

Another issue affecting statistical power is that of multiple testing. Because the number of CVs observed will most certainly be larger than the number of common SNPs observed in any data set, any statistical procedure that tests all CVs must pay a larger “penalty” in terms of correcting for multiple tests, thereby reducing power further.

Several authors have considered the issue of multiple testing and have sought ways to address it in genetic studies [52–57]. One solution is to determine the “true” number of independent SNPs, so that the multiple testing correction is not as egregious.

Ott and colleagues also worked on this problem [58–62]. Specifically, they considered the sums of single-locus statistics, and corrected for multiple testing through permutation. An advantage of this approach is that it incorporates the linkage-disequilibrium (correlation) structure among individual markers. Also, the sum statistic provides a single p-value even though multiple markers are used, so no correction for multiple testing is necessary. This approach has also been incorporated in work by Purcell et al. [63] and Zhou and Pan [64], among others.

In this work, we present a statistic based on the trend test developed by Cochran [65] and Armitage [66]. It is a likelihood ratio test in which the observed data are phenotype and NGS data, specifically the coverage and number of observed CV counts at one or more polymorphic non-synonymous coding SNPs.

We note that work using the trend test for rare variants has already been published [67]. These authors found that in disease models with only rare risk variants, an statistical method based on the Cochran-Armitage trend test had power comparable to or greater than tests that pool (i.e., bin) rare variants. The authors concluded that efficient locus-wide inference using single-variant test statistics should be reconsidered as a useful framework for devising powerful association with rare-variant sequence data. As noted above, in this work we focus on slightly more common CVs.

Our statistic extends the work of Kinnamon et al. [67] in that it probabilistically estimates the true (unobserved) genotype based on the NGS data, and also estimates the sequence misclassification error probabilities in cases and controls (either separately or jointly). We evaluate the performance of our statistic on null and alternative data using simulations. We also use empirical 1000 Genomes SNP data to generate a fictitious disease phenotype, and apply our test statistic to such data to determine the p-value and estimate of various parameters of interest.

Methods

We begin by providing some notation in Table 1. We follow with a description of the test statistic, simulations, and empirical data.

Table 1.

Notation for the LTT_ae,NGS statistic.

Throughout this work, the superscript “t” indicates the true value of a variable. The following notation is in force:

M = The number of loci/sites in the study.

The next several terms are related.

$i_k^t$ = The phenotype code of individual k. The value $i_k^t=1$ means the individual is a case (affected), and the value $i_k^t=0$ means the individual is a control (unaffected).

xk = (x1,k, …, xM,k) = The CV count vector for individual k.

${\mathit{v}}_{\mathit{k}}^{\mathit{t}}=(v_{1,k}^t,\cdots ,v_{M,k}^t)$ = The coverage vector for individual k.

Gt = The vector of true genotypes at the M loci; this vector can take on 3M values, denoted by ( $j_1^t,\cdots ,j_M^t$).

${\mathit{g}}_{(j_1^t,\cdots ,j_M^t)}=Pr\left({\mathit{G}}^{\mathit{t}}=(j_1^t,\cdots ,j_M^t)\right)$, the genotype frequency of the multi-locus genotype Gt in the population.

$f_{i_k^t,{\mathit{G}}^{\mathit{t}}}=f_{i_k^t,(j_1^t,\cdots ,j_M^t)}=Pr\left(i_k^t\mid {\mathit{G}}^{\mathit{t}}(j_1^t,\cdots ,j_M^t)\right)$ = Penetrance of phenotype code $i_k^t$ for individuals with genotype vector ${\mathit{G}}^{\mathit{t}}=(j_1^t,\cdots ,j_M^t)$.

α = Baseline odds-ratio.

β = Log-odds ratio.

wGt = Weight corresponding to the multi-locus genotype Gt. As above, we use dominant weight parameterization. Specifically:

$$w_{{\mathit{G}}^{\mathit{t}}}=\{\begin{array}{l}0,{\mathit{G}}^{\mathit{t}}=(0,\cdots ,0)\hfill \\ 1,\text{otherwise}\hfill \end{array}.$$

α + βwGt = ln(f1,Gt/f0,Gt). NOTE: As with the Single locus association situation, the mixing proportion in the case group is given by:

$f_{1,{\mathit{G}}^{\mathit{t}}}=f_{0,{\mathit{G}}^{\mathit{t}}}\times e^{\alpha +\beta w_{{\mathit{G}}^{\mathit{t}}}}$.

Also, using the notation ${\mathit{G}}^{\mathit{t}}=(j_1^t,\cdots ,j_M^t)$ we have:

$f_{0,{\mathit{G}}^{\mathit{t}}}=\frac{1}{1+e^{\alpha +\beta w_{{\mathit{G}}^{\mathit{t}}}}}$,

so that:

$f_{1,{\mathit{G}}^{\mathit{t}}}=\frac{e^{\alpha +\beta w_{{\mathit{G}}^{\mathit{t}}}}}{1+e^{\alpha +\beta w_{{\mathit{G}}^{\mathit{t}}}}}$.

The null hypothesis for our test statistic is H0:β = 0. Under the null,

$f_{i_k^t,{\mathit{G}}^{\mathit{t}}}=f_{i_k^t,\ast }=\frac{{(e^{\alpha })}^{i_k^t}}{1+e^{\alpha }}$.

${\tau }_{{\mathit{G}}^{\mathit{t}},k}={\tau }_{(j_1^t,\cdots ,j_M^t),k}$ = Conditional probability that individual k has true genotype vector ${\mathit{G}}^{\mathit{t}}=(j_1^t,\cdots ,j_M^t)$, where conditioning is on the observed data xk.

Log-likelihoods and computation of test statistic

Here, we present the log-likelihoods of the observed data, and we state the definition of the linear trend test allowing for errors applied to next generation sequencing (LTT_ae,NGS) in terms of the log-likelihoods.

$$ln(L_{{\mathrm{H}}_d})={\sum }_{k=1}^{N}ln[Pr({\mathit{x}}_{\mathit{k}},{\mathit{v}}_{\mathit{k}}^{\mathit{t}},i_k^t)]=\text{Log}-\text{likelihood}\text{Equation}\text{of}\text{the}\text{observed}\text{data}.$$

Note:

• d = 0 for Null Hypothesis: (H₀ : β^t = 0),

• d = 1 for Alternative Hypothesis: (H₁ : β^t ≠ 0).

ln (L̂_Hd) = Maximum log-likelihood of the data for each hypothesis. This maximum is achieved by applying the EM algorithm in the following way:

1. Specify a certain number of starting points (i.e., randomly generated vectors ψ⃗ of parameter settings for α, β, etc.).

2. For each vector ψ⃗ in Item 1, update the log-likelihoods under each hypothesis until some stopping condition is satisfied, such as:

   $$\mid {(r+1)}^{\text{st}}-\text{step}\text{of}ln(L_{{\mathrm{H}}_d})-{(r)}^{\text{th}}-\text{step}\text{of}ln(L_{{\mathrm{H}}_d})\mid <\delta ,$$ (2)

   for some tolerance δ. In this work, we use δ = 0.00001. The maximum log-likelihood is then the (r)^th - step of ln(L̂_Hd). We denote this value by: ln(L̂_Hd)_r(ψ⃗).

   1. NOTE: For an arbitrary vector ψ⃗ in Item 1, if the stopping condition (2) is not met after the maximum number of steps, we define the log-likelihood as: ln(L̂_Hd)_r(ψ⃗), where r_max is the total number of steps specified for the EM algorithm. For example, in the Simulation section below, r_max = 1000 (Item (xi)).

3. We define the log-likelihood of the observed data, denoted ln (L̂_Hd), as:

   $$ln({\widehat{L}}_{{\mathrm{H}}_d})=\underset{\overrightarrow{\psi }}{max}\left(ln{(L_{{\mathrm{H}}_d})}_{r(\overrightarrow{\psi })}\right).$$

LTT_ae,NGS = 2[ln (L̂_H1) − ln (L̂_H0)]. As noted above in Item (3), the carat symbol ˆ indicates that we have obtained the maximum log-likelihood of the data under the particular hypothesis. Note that LTT_ae,NGS is asymptotically a chi-square distribution with 1 degree of freedom. We design the LTT_ae,NGS statistic to be robust to differential misclassification error among cases and controls; that is, we specify ${\epsilon }_0^t\ne {\epsilon }_1^t$.

Simulations

To evaluate the type I error rate and power of the test statistic under different scenarios, we perform simulations. In this section, we describe simulations where we consider type I error rate and power for a single CV. First, we simulate observed data for a single locus containing a CV for each individual based on their phenotype status. Next, we compute observed data for other loci based on a correlation coefficient.

The parameter settings that we consider for the first locus are:

(i) Disease MOI:	Dominant, Recessive, Multiplicative;
(ii) Prevalence (φ):	0.05, 0.15;
(iii) Genotype Relative Risk (R2):	1.0 (Null), 2.5, 5.0;
(iv) CVF:	0.01, 0.05;
(v) Coverage $v_k^t$ (for all k):	3, 8, 25;
(vi) Control error probability ${\epsilon }_0^t$:	0.001, 0.04;
(vii) Case error probability ${\epsilon }_1^t$:	0.001, 0.04;
(viii) Number of cases:	500;
(ix) Number of controls:	500;
(x) Number of starting points:	200;
(xi) Number of EM steps per starting point:	1000;
(xii) Tolerance ε :	10−5;
(xiii) Number of replicates per vector ((i)–(iv)):	500 (Null), 250 (Alternative).
(xiv) Number of loci:	5, 8;
(xv) Correlation ρ:	0.90.

In item (iii), we specify that the disease locus is in Hardy Weinberg Equilibrium. Also, Genotype relative risk is defined in terms of the penetrances f_i, 0 ≤ i ≤ 2 and the disease MOI. More specifically, f_i = Pr (affected|i copies of the CV at a locus), $R_1=\frac{f_1}{f_0},R_2=\frac{f_2}{f_0}$ [68].

To determine genotypes at the remaining loci the LTT_ae,NGS statistic for multiple loci, we use the correlation coefficient specified in item (xv).

For any consecutive pair of loci, genotype frequencies (and thus CV frequencies) conditional on affection status are computed using the well-known correlation coefficient method for determining linkage disequilibrium. A full description is given in the Appendix.

For the null simulations, using parameter settings (i) – (vii), and noting that Disease MOI reduces to 1 MOI when R₂ = 1, we have a total of 96 simulation vectors. Also, for the alternative simulations, using the same parameter settings and noting that R₂ has only two values, we compute a total of 576 simulation vectors.

We compute the empirical power (see below for null hypothesis term) at different significance levels (5%, 1%) for each of these settings by computing the LTT_ae,NGS for each of the replicates listed in item (xiii), determining the corresponding p-value assuming a central chi-square distribution with 1 degree of freedom under the null hypothesis, and evaluating the proportion of replicates for which the p-value is less than the corresponding significance level. When the null hypothesis is true, we refer to the proportion as the empirical type I error rate for a given vector of parameter settings. When the alternative hypothesis is true, we refer to the proportion as the empirical power for a given vector of parameter settings.

Comparison of LTT_ae,NGS statistic with CMAT and SKAT statistics

We compare our statistic with two other commonly used statistics that test for association with NGS data. They are: (i) the cumulative minor-allele test (CMAT) [69]; and (ii) the sequence kernel association test (SKAT) [70]. We choose these statistics because: (i) they test different components of a compound null hypothesis, as recently documented by Liu et al. (N. Tintle, personal communication; see Acknowledgements); (ii) computation is straightforward, so that we can easily program each statistic; and (iii) each has been well-cited (according to ISI Web of Science).

For the three statistics, we simulate CV count data for multiple loci in cases and controls. We use our statistic to determine (through the EM algorithm) the probabilities of each genotype at each locus. We use this information to determine the genotype probability calculations for SKAT and CMAT (see Appendix). We compute empirical type I error and empirical power values as described above.

Thousand Genomes Data Example

Using information from the most recent 1000 Genome SNP annotation [71], we select five non-synonymous SNPs within a maximum range of 20 kilobases (kB) of each other for multiple persons. The SNPs are listed in Table 2. The individuals chosen come from the following populations:

Table 2.

Non-synonymous Chromosome 20 SNPs selected for multi-locus power calculation.

SNP	POS	REF	ALT
rs2274934	60897487	C	T
rs3737137	60897721	C	T
rs6143021	60897772	T	C
rs3810548	60905878	A	G
rs13042941	60908969	T	C

Column Headings:

SNP = Single Nucleotide Polymorphism ID.

POS = Position of SNP on Chromosome 20.

REF = Reference allele/variant.

ALT = Alternative allele/causal variant.

1. ASW (African Ancestry in Southwest US);

2. CEU (Utah residents (CEPH) with Northern and Western European ancestry);

3. FIN (Finnish natives);

4. GBR (British in England and Scotland);

5. TSI (Toscani in Italia);

6. YRI (Yoruba in Ibadan, Nigeria).

We remove all individuals related to the probands of our data set, so that all individuals in our remaining data set are unrelated. Next, we download these unrelated individuals’ sequenced raw file of exome alignment from the 1000 Genomes Project FTP site [72]. The region of interest we specify (Chromosome 20: Base pair positions 60,897,487-60,908,969) is extracted from each individual by applying SAMtools [73]. This region is chosen because Chromosome 20 is a relatively small chromosome and the corresponding data sets are likewise much smaller and easier to process, computationally.

With the ‘mpileup’ function in SAMtools, we align the extracted region to the reference human genome Build 37 [74]. Then we perform SNP calling using BCFtools. Following this protocol, we obtain coverage and the number of CV reads for each of the 5 SNP positions. Finally, we filter individuals one last time by removing each individual whose coverage at any of the SNPs is 1 or less. We obtain a final set of 388 unrelated individuals.

Next, we create a fictitious disease phenotype as follows:

1. For each individual’s vectors of coverage ${\mathit{v}}_{\mathit{k}}^{\mathit{t}}=(v_{1k}^t,\dots ,v_{5k}^t)$ and CV reads x_k = (x_1k, …, x_5k), we compute a new vector, the proportion of each SNP’s coverage that consists of CV reads. In mathematical terms, we create ${\mathit{U}}_{\mathit{k}}=(u_{1k},\dots ,u_{5k})=(\frac{x_{1k}}{v_{1k}^t},\dots ,\frac{x_{5k}}{v_{5k}^t})$.

2. For each individual, we compute the linear combination $L_k={\sum }_{m=1}^5{b}_m{u}_{mk}$. The weights we specify are: b₁= −1.0, b₂ = 0.5, b₃ = 0.0, b₄ = 8.0, b₂ = 5.0. These weights are selected to provide a range of effects. For example, weight b₁ is −1.0, indicating that the CV is protective, which is contrary to our stated specification. Weights b₂ and weight b₃ are 0.5 and 0.0 respectively, which means they have little or no effect on disease risk. Weights b₄ and b₅ are rather large, suggesting that, if we find association, it is due to effects of SNPs 4 and 5.

3. Compute the median L_k value over all 388 individuals. Define as “affected” any individual whose L_k value is above the median, and define as “unaffected” any individual whose L_k value is below the median. In this way, we obtain exactly 50% cases and 50% controls.

Finally, we apply the LTT_ae,NGS to this data set and compute significance, as well as parameter estimates.

Results

Comparison of LTT_ae,NGS statistic with CMAT and SKAT statistics

Simulations - Null hypothesis

Here, we present results of the null simulations. Specifically, we report summary statistics for the empirical type I error rates with different settings of coverage ( $v_k^t$). These results may be found in Figure 1 (results presented in box and whisker plots [75]). In this figure, the empirical type I error rates are computed only for the differential misclassification simulation settings ( ${\epsilon }_0^t\ne {\epsilon }_1^t$) at the 5% significance level (results for the 1% level were very similar and are not presented). Each box and whiskers plot is computed from 16 empirical type I error rates. The number 16 comes from 2 settings each of: (i) φ; (ii) CVF; (iii) Number of loci; and (iv) differential error rates ( ${\epsilon }_0^t\ne {\epsilon }_1^t$).

Fig. 1.

Box and whiskers plots of empirical type I error rates for LTT_ae,NGS, CMAT, and SKAT statistics with different coverage settings. Each box and whiskers plot is determined using empirical type I error rates for 16 simulations (2 settings each for: (1) number of loci; (2) Φ; (3) CVF; (4) ${\epsilon }_0^t\ne {\epsilon }_1^t$; and 3 settings for v^t (see notation in Methods – Simulations)).

One finding is immediate. The LTT_ae,NGS statistic appears to maintain correct type I error rate for all coverage settings in the presence of differential misclassification. The vector values (1^st Quartile, Median, 3^rd Quartile) for coverages $v_k^t=3,8,25$ are: (0.050, 0.058, 0.066), (0.055, 0.060, 0.067), (0.051, 0.060, 0.072), respectively, for the LTT_ae,NGS statistic. By contrast, for the same coverage ordering, the CMAT vector values are: (0.913, 0.921, 0.975), (0.341, 0.353, 0.367), (0.048, 0.053, 0.059). Likewise, the SKAT vector values are: (0.918, 0.935, 0.964), (0.340, 0.364, 0.374), (0.051, 0.057, 0.063). One may see all these quantities in Figure 1.

Note that, when the coverage is 3, the non-differential statistics have empirical type I error rates that are approximately 18 times the desired rate (0.05). More worrisome, from our perspective, is the fact that the median empirical type I error rates for these statistics are near 1.00; that is, one would almost always reject the true null hypothesis if simulation settings were correct.

When the coverage is 8, the non-differential statistics have empirical type I error rates that are approximately 7 times the desired rate (0.05). While the empirical type I error rates for this coverage are much closer to the 5% significance level, they still indicate substantial inflation in type I error.

Only when the coverage rises to 25 do the empirical type I error rates for all three statistics appear to approximately equal to the 5% significance level.

These results strongly suggest that, at lower coverage rates, not allowing for differential misclassification may result a (sometimes substantial) increase in type I error for an NGS statistic. We note that a non-differential form of our LTT_ae,NGS statistic produced similar inflation in empirical type I error to the CMAT and SKAT statistics (results not shown).

As previous research has documented [11,26,30,36,76–78], when applying a statistic test that does not account for the potential differential genotype misclassification error, we observe an inflated empirical type I error rate. This makes sense because, unless we account for the differential errors in the analysis, the differential errors manifest themselves as a true phenotype-genotype association. Another question is raised by studying Figure 1: why is there substantial inflation in empirical type I error for lower coverage, but none observed for higher coverage? This question may be answered by observing that we specify the probability of the observed CV counts x_m,k conditional on the underlying true genotype (and other parameters) as a binomial distribution (Appendix, section entitled “Component probability mass function for Causal Variant (CV) count data for single locus”. Consider the values: $v_k^t=3$, j^t = 0 genotype (homozygous for the non-causal variant allele), and misclassification rate ε_j^t = 0.04. The probability of an observed count x_k = 1 is 11% (we drop the m subscript from x to indicate we are considering data for a single variant). If we consider the heterozygote : j^t = 1, the probability of an observed count x_k =1 is 37.5%. From these values, we see that it is somewhat difficult to determine with certainty the true underlying genotype.

Now consider the values: $v_k^t=25$, j^t = 0 genotype, and misclassification rate ε_j^t = 0.04. The probability of an observed count x_k = 1 is 37.5%. For the heterozygote : j^t = 1, the probability of an observed count x_k = 1 is 7.45E-05% (virtually 0). Also, for the j^t = 2 genotype, the probability is 6.76E-31% (we would call this value 0). From these results, we would conclude with near certainty that the underlying genotype is j^t = 0. Hence, even in the presence of misclassification error, higher coverage enables us to determine (much more precisely) the true underlying genotypes. It is for this reason that all three statistics maintain a correct type I error rate when $v_k^t=25$.

Finally, we note that, for the non-differential error simulations ( ${\epsilon }_0^t={\epsilon }_1^t$), all three statistics demonstrate correct empirical type I error rates at all coverages (results not shown). Therefore, we present empirical power results only for the non-differential settings.

Simulations - Alternative hypothesis

In Table 3, we present summary statistics of the empirical power differences (CMAT - LTT_ae,NGS) and (SKAT - LTT_ae,NGS) at the 5% significance level. Results are stratified by the Disease MOI. Studying this table, we observe that the median and mean empirical power differences are close to 0 for nearly all Disease MOI and statistics. Similarly, the mean differences are also close to 0.

Table 3.

Summary statistics for empirical power differences (CMAT - LTT_ae,NGS) and (SKAT - LTT_ae,NGS).

Statistic	Disease MOI	Min	1st Q	Median	Mean	3rd Q	Max
CMAT	Dominant	−0.096	0.000	0.000	0.004	0.004	0.100
	Multiplicative	−0.080	−0.004	0.010	0.012	0.032	0.076
	Recessive	−0.040	−0.012	0.004	0.014	0.030	0.104
SKAT	Dominant	−0.148	0.000	0.000	0.009	0.004	0.144
	Multiplicative	−0.116	0.000	0.016	0.022	0.048	0.104
	Recessive	−0.047	−0.008	0.008	0.016	0.028	0.112

The column heading abbreviations corresponding to each statistic and Disease MOI setting are defined as follows:

Min = Minimum power difference.

1^st Q = 1^st Quartile value for the power difference.

Median = Median power difference.

3^rd Q = 3^rd Quartile value for the power difference.

Max = Maximum power difference.

All empirical powers were computed at the 5% significance level only for non-differential error simulations.

Another observation is that the power differences are slightly skewed in favor of the CMAT and SKAT statistics. Specifically, the 3^rd Quartile values are all larger than the corresponding 1^st Quartile values for all rows. Having said this, we note that the minimum difference (in absolute value) is close to, if not greater than the corresponding maximum value for each row, with the exception of the Recessive Disease MOI. We conjecture that power differences are “more positive” for the Recessive MOI because our statistic is really a “dominant weight” statistic [79]; that is, any multi-locus genotype containing at least one CV is given a weight of 1, and the only multi-locus genotype with a 0 weight is the one containing no CVs.

Because the empirical powers of the three methods were so similar, we focus on power results for the LTT_ae,NGS statistic. We ran an analysis of variance (ANOVA) as implemented in the R software package [80] to determine what factors most significantly affect empirical power at the 5% level (results for 1% significance level nearly identical and therefore not shown). The factors we considered as input variables were items (i) – (viii) and (xiv) listed in the Methods section (Simulations) and all two-way interactions. The response variable was the empirical power corresponding to the specific simulation vector. Note that, because the LTT_ae,NGS statistic is robust to differential misclassification, we include empirical power results for differential error as well.

Here, we present just the terms of the ANOVA whose F statistic had a p-value less than 2.00E-16 (full results not shown). In order of their F-statistic values (largest to smallest), the most significant factors are: (i) Disease MOI (F = 13119); (ii) CVF (F = 2372); (iii) R₂ (F = 1607); (iv) Disease MOI × CVF (F = 928); (v) Disease MOI × R₂ (F = 488); and (vi) CVF × R₂ (F = 164).

To illustrate the importance of the first two factors (Disease MOI and CVF) on empirical powers, we present a box and whiskers plot of the powers, stratified by settings for these two factors (Figure 2). Examining this figure, we observe that minimum, median, and maximum power can vary substantially, depending upon the Disease MOI. For example, the minimum power for the Dominant MOI is 0.5, when the CVF is 0.01. By contrast, the maximum power for the Recessive MOI is 0.14, when the CVF is 0.05.

Fig. 2.

Box and whiskers plots of empirical powers for LTT_ae,NGS for different settings of disease MOI and CVF. Each box and whiskers plot is determined using empirical powers for 96 simulations (2 settings each for: (1) number of loci; (2) Φ; (3) R₂; (4) ${\epsilon }_0^t$; (5) ${\epsilon }_1^t$; and 3 settings for v^t (see notation in Methods – Simulations)). Dom = Dominant; Mult = multiplicative; Rec = recessive disease MOI, respectively.

It is also reasonably clear that CVF makes a difference when considering Disease MOI. For example, when the CVF is 0.01, Dominant MOI most empirical powers range from 0.70 (1^st Quartile) to 1.00 (3^rd Quartile). By contrast, when the CVF is 0.05, power is virtually 1.00 for all other settings. Also, for the Multiplicative MOI, when CVF is 0.01, empirical powers ranges primarily from 0.20 (1^st Quartile) to 0.55 (3^rd Quartile). However, when the CVF is 0.05, empirical powers increase so that the range is mostly 0.67 (1^st Quartile) to 0.99 (3^rd Quartile).

Thousand Genomes Data Example

When running the LTT_ae,NGS on the 1000 Genomes data set, we obtain a test statistic value of 42.643, with a corresponding p-value of 6.57E-11. The final parameter estimates are: α̂ =−13.144, β̂ =13.334, $\widehat{{\epsilon }_0}=0.017,\widehat{{\epsilon }_1}=0.014$, under the alternative. The genotype frequencies are not listed, since there are 243 such frequencies. The carat above each parameter indicates that this is a maximum likelihood estimate. It is interesting to note that the error values are independent of the fictitious disease model; that is, it appears that there is an approximately 1.5% probability of misclassifying sequence data in the 1000 Genomes Project for these individuals and SNPs. Finally, we note that, for our disease model, the increase in risk of getting the disease is e^{(−13.144+13.334)} = 1.21 for any individual harboring a multi-locus genotype with at least one CV.

Summary and Discussion

The purpose of this work is to develop a linear trend test that can utilize phenotype and NGS data to perform a case-control genetic association analysis. As mentioned in the Introduction and documented in the Appendix, our statistic probabilistically estimates the true (unobserved) genotype based on the NGS data, and also estimates the sequence misclassification error probabilities in cases and controls separately. Based on the results of simulations, our statistic appears to maintain the correct type I error rate in the presence of either differential or non-differential misclassification. Also, empirical power appears to be comparable to statistics such as the CMAT and SKAT statistics, depending upon the GRR and CVF. One interesting result from the 1000 Genomes SNP data set is that our statistic estimates sequence misclassification errors of approximately 1.5% for the SNPs sequenced.

One of the key findings in this work is that the development of such a statistic is indeed possible; in the Appendix, we present closed-form solutions of all necessary parameters. From this information, we may be able to extend the statistic to other models that are binary, in the sense that weights are either 0 or 1. We performed such computations for our linear trend test using double sampling [32].

In previous research, we found that phenotype misclassification error has much larger effect, in terms of power loss, than genotype misclassification error. The one kind of genotype misclassification error that can be vexing is differential genotype misclassification. As noted in the Introduction, and as documented in our Results (Figure 1), differential genotype misclassification, even at low rates, can produce substantially inflated type I error rates [11,26,30,36,78], so that the null distribution of the test statistic is no longer known. For that reason, we recommend that statistics like LTT_ae,NGS be used when testing for association or, that already developed NGS statistics be modified to allow for differential misclassification. This recommendation is particularly important for designs with low coverage. In results not shown, even when error is non-differential, there is practically no power loss for the LTT_ae,NGS statistic as compared with a non-differential form of the same statistic, so we suggest always using the statistic we report in this work. Such a suggestion is analogous to recommendations that one use the H-LOD statistic (as compared with the LOD statistic) when performing linkage analysis, or that one use the t-test allowing for unequal variance as compared with the one that assumes equal variance in two groups; that is, these statistics are robust to some deviation from a common assumption (e.g., linkage homogeneity, equal variance) and there is very little power loss when using these statistics.

We are aware that the format of NGS data is evolving. The latest set of data from the 1000 Genomes Project has actual genotype calls along with quality scores. One of the nice features of likelihood based methodology is that, as long as probabilities can be determined, statistics can be computed for multiple formats. We look forward to developing such statistics.

As mentioned in the Introduction, we are keenly aware of the clinical importance regarding identification of very rare variants. While the focus this work has been an evaluation of the LTT_ae,NGS statistic applied to more common variants (1% ≤ CVF ≤ 5% for each CVF), we note that our statistic can handle any CVFs and determine accurate results. The reason is that the asymptotic distribution of the test statistic is more accurate as the number of loci considered is increased (irrespective of the CVFs), needing less reliance on permutation methods to compute significance. Also, our statistic was designed to handle multiple CVs, since increase in the number does not affect the degrees of freedom.

Is it possible to determine differential misclassification without the application of our statistic? We think so. In fact, we are aware of at least one paper [78] that looked at Q-Q plots to determine differential misclassification. If researchers have a sufficient number of polymorphic sites for which there is NGS data, we recommend using Q-Q plots.

Another diagnostic test involves checking whether the missing rates among genomic sites differ between cases and controls [82]. While neither of these methods is definitive in determining differential misclassification (only double-sampling [82] can provide near perfect evidence), results that deviate from expected results can be strongly suggestive.

We note that not all researchers will perform either whole-exome or whole genome NGS studies; some may perform candidate region studies with a relatively small number of sites tested. In those instances, Q-Q plots may not be as helpful, as they may not have a sufficient number of data points.

In either situation, we comment that results from our simulations suggest that the LTT_ae,NGS is robust to differential misclassification. Therefore, regardless of the nature of the misclassification, we anticipate that our statistic will provide valid results.

Along with the comments above, our statistic may be used in a confirmatory analysis for researchers who have found significant association with other methods. In this way, researchers can protect themselves from following up false positives.

Finally, we have developed an R version of the analysis software. In addition, we are developing a C version of the software for results using permutation testing. We plan to make these items available on the web by May 2013 to any interested researchers.