Detecting genes in complex disease: does phenotype accuracy limit the horizon?

In their recent article, Samuels et al. [1] suggested that diagnostic error rates of just a few percent have such a detrimental effect on power that massive increases in sample size would be required to offset their adverse effects. The authors therefore concluded that the accuracy of phenotyping is a major factor limiting the ability of association studies to identify novel genes in complex traits. We are writing to point out that they have seriously over estimated the influence of this source of error; recalculation shows that even in the worst case scenario diagnostic error rates would have to be higher than 10% before they are likely to have any substantial effect on power.

No diagnostic process is perfect and no matter how careful researchers are it is inevitable that some individuals who are unaffected will be mistakenly labelled as cases and vice versa. The PAWE program [2] allows researchers to calculate the power to detect association in the face of these diagnostic errors. Within the PAWE program error rates are specified in terms of the conditional probabilities Pr(observed control | true affected) (the proportion of truly affected individuals mistakenly labelled as unaffected) and Pr(observed case | true unaffected) (the proportion of truly unaffected individuals mistakenly labelled as affected). In using this program it is important to remember that the diagnostic error rate is the proportion of individuals labelled as affected that are actually unaffected i.e. the conditional probability Pr(true unaffected | observed case) and that in general:

$$\text{Pr}(\text{true unaffected}|\text{observed case})\ne \text{Pr}(\text{observed case}|\text{true unaffected})$$

Unfortunately in their analysis of the relationship between power and diagnostic error rate, Samuels et al. [1] seem to have mistakenly used the diagnostic error rate as the input for the PAWE program and have thereby substantially overestimated the impact of diagnostic errors on power. For a given prevalence and diagnostic sensitivity, straight forward algebra enables the correct input for PAWE, Pr(observed case | true unaffected), to be calculated for any given diagnostic error rate, Pr(true unaffected | observed case). Here, we reproduce the main figure from Samuels et al. [1] after calculating Pr(observed case | true unaffected) on the basis of Pr(true unaffected | observed case) as should have been done (Figure 1). These corrected curves show that diagnostic errors have almost no impact on power when the sample size employed is inadequate or excessive with respect to the size of effects sought, and only limited impact even when sample size is critical. The figures show that even in the worst case scenario diagnostic error rates as high as 10% reduce power by little more than 20%. By correcting the simple arithmetic error undermining the results presented by Samuels et al. [1] provides a more accurate illustration of how diagnostic error impacts on power which is in line with our previous report on this issue [3]. While it is clear that any diagnostic error reduces power it is also clear that the impact is not as great as has been suggested. In short our answer to the question posed by Samuels et al. [1] in the title of their article would be “yes, but not by as much as you might think”.

Figure 1.

Power to detect a genetic association in the context of diagnostic errors. In each example, the probability of affected individuals being classified as controls is 1 × 10⁻⁵. Varying this parameter has negligible impact on power and/or optimal sample size for diseases that are present in less than 10% of the population. (a) Power to detect an association between a common allele (allele frequency = 0.5; GRR = 1.1– 1.3 under a multiplicative model) and disease in 20000 cases and 20000 controls with varying degrees of diagnostic error at P < 5 × 10⁻⁷. Disease frequency = 0.01. (b) Power to detect an association between alleles of different frequency (0.5, 0.25, 0.1) and disease in 20000 cases and 20000 controls with varying degrees of diagnostic error at P < 5 × 10⁻⁷. GRR = 1.3, disease frequency = 0.01. (c) Power to detect an association between an allele (frequency = 0.125, GRR = 1.3) and diseases of different prevalence (0.01, 0.001, 0.0001) in 20000 cases and 20000 controls with varying degrees of diagnostic error at P < 5 × 10⁻⁷. (d) Ratio of the number of inaccurately phenotyped cases (n_error) to the number of accurately phenotyped cases (n_{no error}) required to detect an association between an allele (frequency = 0.1, varying GRR from 1.1 to 1.3) and a disease (frequency = 0.01) with 95% power at varying degrees of diagnostic error at P < 5 × 10⁻⁷. All calculations used PAWE-PH Phenotype Edition [2].