Algorithm Overview 4: Anders and Huber’s⁸ DESeq

The read count ygi is modeled by a GLM of negative binomial distribution with a log link: $$\log ({\lambda }_{gi})={\displaystyle \sum _{p=1}^P{x}_{ip}{\beta }_{gp}}$$ (3.2.c) The mean µgi is the proportion of reads for gene g in sample i, λgk(i), scaled by a normalization factor, mi. The variance ${\sigma }_{gi}^2$ is ${\mu }_{gi}+m_i^2{\nu }_{gk(i)}$, where νgk(i) is assumed to be a per gene raw variance, a smoothing function of λg and k. The use of the smoothing function can help stabilize the variance estimates especially when the number of samples is small. For the estimation of the normalization factor (which is referred to as the size factor by Anders and Huber), mi, for each sample, the authors noted that highly DE genes are more likely to be influential on total count and so the median of the ratios of counts should be used for more robustness: $${\widehat{m}}_i=\underset{i}{\text{median}}\frac{y_{gi}}{{\left({\displaystyle \prod _{v=1}^N{y}_{gv}}\right)}^{1/N}}$$ (3.2.d) Since λgk(i) is proportional to the expected value of the unknown proportion from gene g in group k, it is estimated by the average of counts from all samples in group k with a common scale. $${\widehat{\lambda }}_{gk(i)}=\frac{1}{M_k}{\displaystyle \sum _{i:k(i)=k}\frac{y_{gi}}{{\widehat{m}}_i}},$$ (3.2.e) where Mk is the total number of replicates for group k. The sample variances with the common scale are calculated as: $$w_{gk}=\frac{1}{M_k-1}{{\displaystyle \sum _{i:k(i)=k}\left(\frac{y_{gi}}{{\widehat{m}}_i}-{\widehat{\lambda }}_{gk(i)}\right)}}^2$$ (3.2.f) $$z_{gk}=\frac{{\widehat{\lambda }}_{gk(i)}}{M_k}{\displaystyle \sum _{i:k(i)=k}\frac{1}{{\widehat{m}}_i}}$$ (3.2.g) In the case of a sufficiently large number of Mk, one can see wgk – zgk as the unbiased estimator of the raw variance νgk. In the case of a small number of Mk, local regression for a smooth function wk(λ) on the graph of $({\widehat{\lambda }}_{gk(i)},w_{gk})$ was suggested so that $w_k({\widehat{\lambda }}_{gk(i)})-z_{gk}$ would be the estimate for the raw variance. More details are in the study by Anders and Huber.8