Version Changes
Revised. Amendments from Version 1
We would like to thank the reviewers for their excellent reviews and valuable feedback. We have taken into consideration all of the reviews and comments, and have improved the software, updated the software vignettes, and added more figures to the manuscript. In particular we
merged David Robinson's pull request, added the customCode argument, provide code for making histograms instead of density plots which is used in one of the example use cases, and requested that quotation marks are typeset in a way that makes them easy to copy-paste into R;
made it easier to use and install dependencies via a new function load_install() based on Karthik Ram's feedback, added a link to the bibliography file;
modified the manuscript and added more example use cases to motivate users to use regionReport, particularly with DESeq2 or edgeR results which expands the software's usefulness to a broader user base.
This manuscript matches the software release from Bioconductor version 3.3 which uses by default rmarkdown's version 0.9.5 capabilities. We considered using git2r and have not added support for so far given that (A) we mostly use regionReport in git-controlled directories that are not on the gh-pages branch and (B) that some of the reports can be large in size. However, we might add a function for uploading the resulting report to GitHub via git2r in the future.
Abstract
regionReport is an R package for generating detailed interactive reports from region-level genomic analyses as well as feature-level RNA-seq. The report includes quality-control checks, an overview of the results, an interactive table of the genomic regions or features of interest and reproducibility information. regionReport provides specialised reports for exploring DESeq2, edgeR, or derfinder differential expression analyses results. regionReport is also flexible and can easily be expanded with report templates for other analysis pipelines.
Keywords: Report, Interactive, Reproducibility, Genomics, Sequencing, ChIP-seq, RNA-seq, Software
Introduction
Many analyses of genomic data result in regions along the genome that associate with a covariate of interest. These genomic regions can result from identifying differentially bound peaks from ChIP-seq data 1, identifying differentially methylated regions (DMRs) from DNA methylation data 2, or performing base-resolution differential expression analyses using RNA sequencing data 3, 4, among other analysis pipelines. The genomic regions themselves are commonly stored in a GRanges object from GenomicRanges 5 when working with R or the BED file format on the UCSC Genome Browser 6. Other information on these regions, for example summary statistics on the magnitude of effects and statistical significance, also provide useful information and can be stored as metadata in GRanges objects. The usage of R in genomics is increasingly common due to the usefulness and popularity of the Bioconductor project 7, and in the latest version (3.3), 300 unique packages use GenomicRanges for many workflows, demonstrating the widespread utility of identifying and summarizing characteristics of genomic regions.
Bioconductor is particularly strong for differential expression analyses, with 206 packages using the Differential Expression BiocView. RNA-seq data is commonly used to perform feature-level analyses at either the transcript, gene or exon levels with Bioconductor packages DESeq2 8 and edgeR 9– 11, among others. The features can also be expressed regions identified in an annotation-agnostic procedure by derfinder 3. In an exploratory data analysis of DESeq2 or edgeR results it is common to create a set of plots in order to identify potentially problematic samples or features. For example, in such an exploratory analysis it is common to use a dimension reduction technique such as principal component analysis to determine if samples are clustering by group or another variable of interest. This type of plot is useful for detecting artifacts, such as mislabeling of samples.
Here we introduce regionReport which allows users to explore genomic regions of interest, derfinder, DESeq2, and edgeR results through interactive stand-alone HTML reports that can be shared with collaborators. These reports are flexible enough to display plots and quality control checks within a given experiment, but can easily be expanded to include custom visualizations or text describing the main conclusions of the exploratory analysis. The resulting HTML report emphasizes reproducibility of analyses 12 by including all the R code without obstructing the resulting plots and tables. Alternatively, static PDF reports can be generated and easily shared among collaborators. We envision regionReport will provide a useful tool for exploring and sharing genomic region-based, DESeq2, and edgeR results from high throughput genomics experiments.
Methods
Implementation
The package includes R Markdown templates which are processed using rmarkdown 13 and knitr 14 to produce HTML or PDF reports. HTML reports can be styled using knitrBootstrap 15 or with rmarkdown templates that include interactive features. The regionReport package generates a report that includes a series of plots for checking the quality of the results and an interactive table with the best regions or features. Each element of the report has a brief explanation, although actual interpretation of the results is dataset- and workflow-dependent. To facilitate navigation a menu is included, which is useful for users interested in a particular section of the report. Figure 1A shows the menu of the general report for a set of regions with associated p-values. The code for each plot or table is hidden by default and can be shown by clicking on the “code” button as shown in Figure 2. Further customization of the reports can be done by providing custom code, changing the default plots, or by modifying the R Markdown templates included in regionReport.
Figure 1. regionReport overview.
Example region input, the appropriate regionReport function to use, and menu of the resulting report for: (A) the general use case, (B) a customised report, (C) derfinder results, (D) DESeq2 results and (E) edgeR results.
Figure 2. Interactively display the code for each table/figure in the report.
(AB) View by default and (B) after clicking on the “code” toggle for a section in the report and the HTML reports include a toggle to hide/show all the R code.
General region report
Quality checks
This section of the report includes a variety of quality control steps which help the user determine whether the results are sensible. The quality control steps explore:
P-values, Q-values, and FWER adjusted p-values
Region width
Region area: sum of single-base level statistics (if available)
Mean coverage or other score variables (if available)
A combination of density plots and numerical summaries are used in these quality checks. If there are statistically significant regions, the distributions are compared between all regions and the significant ones. For example, the distribution region widths might have a high density of small values for the global results, but shifted towards higher values for the subset of significant regions as shown in Figure 3.
Figure 3. Distribution of region widths for all regions in the derfinder use case example with the BrainSpan dataset.
The top figure shows the region width distribution for all regions while the bottom one shows it only for the significant regions. One line is shown per chromosome in each of the plots.
Genomic overview
The report includes plots to visualize the location of all the regions as well as the significant ones. Differences between them can reveal location biases. The nearest known annotation feature for each region is summarized and visually inspected in the report. This type of plot can be useful to quickly check whether significant regions are concentrated in a chromosome or in an annotation type. For example, Figure 4 shows the annotation information for the significant regions with most regions contained inside genes, which is expected with RNA-seq data.
Figure 4. Genomic overview of the annotation type for the significant regions in the derfinder use case example with the Hippo dataset.
Best regions
An interactive table with the top regions (500 by default) is included in this section as shown in Figure 5A. This allows the user to sort the region information according to their preferred ranking option. For example, lowest p-value, longest width, chromosome, nearest annotation feature, etc. The table also allows the user to search and subset it interactively as shown in Figure 5B. A common use case is when the user wants to check if any of the regions are near a known gene of their interest.
Figure 5. Interactive table with results for the top regions in the general use case example using bumphunter results.
The interactive table can ( A) show all the top regions of ( B) a subset of the results by using the search box. The table can also be sorted by each of the different columns.
Reproducibility
At the end of the report, detailed information is provided on how the analysis was performed. This includes the actual function call to generate the report, the path where the report was generated, time spent, and the detailed R session information including package versions of all the dependencies. An example is shown in Figure 6 with the R package information truncated.
Figure 6. Reproducibility section for a report using DESeq2 results.
The reproducibility information includes the actual function call used to generate the report, the path where the report was generated, the time it took to create the report, details about the R session information, and the pandoc version used for rendering the HTML report. For reports based on DESeq2 results, the version used to perform the differential expression analysis and cutoff used are also displayed. Note that DESeq2 version used for the analysis and for the report might differ.
The R code for generating the plots and tables in the report is included in the report itself, thus allowing users to manually reproduce any section of the report, customize them, or simply change the graphical parameters to their liking.
Customization
regionReport allows users to customize the reports to their liking. This can be done in different ways depending on the amount of customization the user is looking for. Several plots are made with ggplot2 and the user might want to change the default theme, for example to a black and white theme as shown in the function call in Figure 6. Another user might be interested in adding code that creates more plots than the ones included by default in the report. For example, the user might be interested in adding a MA and a PCA plot to the default report. This can be done via the customCode argument which results in new sections added to the menu as shown in Figure 1B compared to Figure 1A. Further customization can be achieved by modifying the templates included in regionReport and using the template argument.
derfinder report
When exploring derfinder results from the single base-level approach, for each of the best 100 (default) DERs a plot showing the coverage per sample is included in the report. These plots allow the user to visualize the differences identified by derfinder along known exons, introns and isoforms. The plots are created using derfinderPlot 16. Due to the intrinsic variability in RNA-seq coverage data or mapping artifacts, in situations where there are two candidate DERs that are relatively close there might be reasons to consider them a single candidate DER and its important to visualize them. This tailored report groups candidate DERs into clusters based on a distance cutoff. After ranking them by their area, for the top 20 (default) clusters it plots tracks with the coverage by sample, the mean coverage by group, the identified candidate DERs colored by whether they are statistically significant, and known alternative transcripts as shown in Figure 7. Figure 1C shows the main categories of the report generated from a richer region data set than in the general case.
Figure 7. Example region cluster plot for the derfinder use case example with the BrainSpan dataset.
Coverage curves are shown for each sample colored by their group membership. Mean coverage curves by group, differentially expressed regions (DERs) and known transcripts are shown in the remaining tracks.
DESeq2 and edgeR reports
Feature-level differential expression analyses result in a set of features (genes, exons) with a p-value for each feature. To perform such analyses, some phenotype information about the samples is usually available. With this information, you can explore the raw data to identify potentially problematic samples using principal component analysis and sample distance plots. You can also explore the results and check the features marked as differentially expressed with MA plots and a histogram of the p-values distribution. regionReport provides a template that allows you to create all these plots easily for DESeq2 results ( Figure 1D). It has similar components to the region-level reports such as an interactive table for the top features as shown in Figure 8, but also highlights specific exploratory plots for this type of results. regionReport can also be used for edgeR results ( Figure 1E) resulting in very similar reports given the internal implementation. The only difference is that reports for edgeR results include sections for visualizing the biological coefficient of variation and the multidimensional scaling plot of distances between feature expression profiles. See the use cases for example reports from DESeq2 and edgeR results.
Figure 8. Interactive table for top features from the DESeq2 use case example.
Operation
Installation. regionReport and required dependencies can be easily installed from Bioconductor with the following commands:
source(“ http://bioconductor.org/biocLite.R”)
biocLite(“ regionReport”)
Input. To generate the report, the user first has to identify the regions of interest according to their analysis workflow. For example, by performing bumphunting to identify DMRs with bumphunter. The report is then created using renderReport() which is the main function in this package as shown in Figure 1A,B.
For the derfinder use case, the derfinderReport() function creates the recommended report that includes visualizations of the coverage information for the best regions and clusters of regions. Similarly DESeq2Report() and edgeReport() create reports for DESeq2 and edgeR results, respectively.
Output. A small example can be generated using:
example(“ renderReport”, “ regionReport”, ask=FALSE)
The resulting HTML file will open in the users default browser when using R in an interactive session. Note that alternative output formats such as PDF files can also be generated, although they are not as dynamic and interactive as the HTML format.
Use cases
The supplementary website contains reports using DiffBind, bumphunter, derfinder, DESeq2, and edgeR results. The derfinder use case is illustrated with data sets previously described 3 with a moderately sized data set (25 samples), and a large data set with 484 samples. We encourage you to explore the following example reports:
general HTML report example using bumphunter results,
customized general HTML report using DiffBind results with histograms instead of density plots,
edgeR HTML and PDF reports using the custom ggplot2 theme theme_linedraw(),
edgeR-robust HTML report,
HTML report using derfinder results with the BrainSpan dataset (484 samples) and styled with knitrBootstrap,
HTML report using derfinder results with the Hippo dataset (25 samples) and styled with knitrBootstrap.
Summary
regionReport creates interactive reports from a set of regions and can be used in a wide range of genomic analyses. Reports generated with regionReport can easily be extended to include further quality checks and interpretation of the results specific to the data set under study. These shareable documents are very powerful when exploring different parameter values of an analysis workflow or applying the same method to a wide variety of data sets. The reports allow users to visually check the quality of the results, explore the properties of the genomic regions under study, and inspect the best regions and interactively explore them.
Furthermore, regionReport promotes reproducibility of data exploration and analysis. Each report provides R code that can be used as the starting point for other analyses within a dataset. regionReport provides a flexible output for exploring and sharing results from high throughput genomics experiments.
Software availability
Software access
regionReport is freely available via Bioconductor at Bioconductor.org/packages/regionReport. The supplementary website http://leekgroup.github.io/regionReportSupp/ hosts the code and output for generating all the use cases described. Versions of all software used are included in the reports.
Latest source code
The latest source code is available at github.com/leekgroup/regionReport. However, we highly recommend users to install regionReport directly from Bioconductor at bioconductor.org/packages/regionReport.
Archived source code as at the time of publication
Archived source code available at dx.doi.org/10.5281/zenodo.55274
License
Artistic-2.0.
Acknowledgements
We would like to acknowledge Michael I. Love for his feedback and input in creating the report specific to DESeq2 results.
Funding Statement
J.T.L. was partially supported by NIH Grant 1R01GM105705, L.C-T. was supported by Consejo Nacional de Ciencia y Tecnología México 351535, AEJ was partially supported by 1R21MH109956.
I confirm that the funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.
[version 2; referees: 2 approved
References
- 1. Stark R, Brown G: DiffBind: differential binding analysis of ChIP-Seq peak data.2011. Reference Source [Google Scholar]
- 2. Jaffe AE, Murakami P, Lee H, et al. : Bump hunting to identify differentially methylated regions in epigenetic epidemiology studies. Int J Epidemiol. 2012;41(1):200–209. 10.1093/ije/dyr238 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3. Torres LC, Frazee AC, Love MI, et al. : derfinder: Software for annotation-agnostic RNA-seq differential expression analysis. bioRxiv. 2015;015370 10.1101/015370 [DOI] [Google Scholar]
- 4. Frazee AC, Sabunciyan S, Hansen KD, et al. : Differential expression analysis of RNA-seq data at single-base resolution. Biostatistics. 2014;15(3):413–26. 10.1093/biostatistics/kxt053 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5. Lawrence M, Huber W, Pagès H, et al. : Software for computing and annotating genomic ranges. PLoS Comput Biol. 2013;9(8):e1003118. 10.1371/journal.pcbi.1003118 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6. Rosenbloom KR, Armstrong J, Barber GP, et al. : The UCSC Genome Browser database: 2015 update. Nucleic Acids Res. 2015;43(Database issue):D670–D681. 10.1093/nar/gku1177 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7. Huber W, Carey VJ, Gentleman R, et al. : Orchestrating high-throughput genomic analysis with Bioconductor. Nat Methods. 2015;12(2):115–121. 10.1038/nmeth.3252 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8. Love MI, Huber W, Anders S: Moderated estimation of fold change and dispersion for RNA-seq data with DESeq2. Genome Biol. 2014;15(12):550. 10.1186/s13059-014-0550-8 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9. Robinson MD, McCarthy DJ, Smyth GK: edgeR: a Bioconductor package for differential expression analysis of digital gene expression data. Bioinformatics. 2010;26(1):139–140. 10.1093/bioinformatics/btp616 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10. McCarthy DJ, Chen Y, Smyth GK: Differential expression analysis of multifactor RNA-Seq experiments with respect to biological variation. Nucleic Acids Res. 2012;40(10):4288–4297. 10.1093/nar/gks042 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11. Zhou X, Lindsay H, Robinson MD: Robustly detecting differential expression in RNA sequencing data using observation weights. Nucleic Acids Res. 2014;42(11):e91. 10.1093/nar/gku310 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12. Sandve GK, Nekrutenko A, Taylor J, et al. : Ten simple rules for reproducible computational research. PLoS Comput Biol. 2013;9(10):e1003285. 10.1371/journal.pcbi.1003285 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13. Xie Y: Dynamic Documents with R and knitr. CRC Press,2013;216 Reference Source [Google Scholar]
- 14. Rstudio: RMarkdown: Dynamic Documents for R.2014. Reference Source [Google Scholar]
- 15. Hester J: Knitr Bootstrap framework, R package.2015. Reference Source [Google Scholar]
- 16. Collado-Torres L, Jaffe AE, Leek JT: derfinderPlot: Plotting functions for derfinder.2015. Reference Source [Google Scholar]