Abstract
Recently proposed tumor fitness measures, based on profiling neoepitopes for reactive viral epitope similarity, have been proposed to predict response to immune checkpoint inhibitors in melanoma and small-cell lung cancer. Here we applied these checkpoint based fitness measures to the matched checkpoint treatment naive Cancer Genome Atlas (TCGA) samples where cytolytic activity (CYT) imparts a known survival benefit. We observed no significant survival predictive power beyond that of overall patient tumor mutation burden, and furthermore, found no association between checkpoint based fitness and tumor T-cell infiltration, cytolytic activity, and abundance (tumor infiltrating lymphocyte, TIL, burden). In addition, we investigated the key assumption of viral epitope similarity driving immune response in the hepatitis B virally infected liver cancer TCGA cohort, and uncovered suggestive evidence that tumor neoepitopes actually dominate viral epitopes in putative immunogenicity and plausibly drive immune response and recruitment.
Subject terms: Tumour immunology, Tumour virus infections, Computational models, Predictive medicine
Introduction
Given the growing need to predict the efficacy of tumor immunotherapy, urgent efforts are being made to quantify tumor fitness modulation by immune selection pressure1,2. Immune-based anti-tumoral response is driven in part by T-cell receptor (TCR) recognition of tumor neoepitopes, which mainly represent mutated protein epitopes (i.e., neoepitopes) derived from somatic mutations. These neoepitopes are presented by host-cell major histocompatibility complex (MHC) molecules to facilitate ‘self’ recognition by TCRs (Fig. 1A). Cancer clones that escape immune recognition can hijack CTLA-4 (cytotoxic T-lymphocyte-associated protein 4) and PD-1 (programmed cell death protein 1) checkpoint blockade, resulting in heterogeneous tumors with a complex immunogenomic profile which may include immune exhaustion, giving rise to mixed or unfavorable response to immune checkpoint inhibition1,3.
Recently, a tumor fitness model has been proposed that incorporates two key components defining the TCR-neoepitope interaction: putative neoepitope immunogenicity and TCR recognition4. Putative immunogenicity is modeled via a nonlinear function of in silico derived MHC-I binding affinities arising from the clonal somatic mutation spectrum, while the TCR recognition likelihood is set to scale with sequence similarity to known, infectious, virally derived epitopes from the Immune Epitope Database (IEDB, www.iedb.org)5. Given these assumptions, a combined measure of tumor fitness is then defined as the inverse of the maximum clonal immunogenicity potential, Ϊ, weighted across subclones in the tumor (see Methods, Fig. 1B).
Although this model successfully predicted response to checkpoint inhibition in melanoma and small cell lung cancer, it is of interest to investigate its applicability, positive predictive power, and key assumptions in the arena of analogous immunotherapy naive (endogenous) TCGA skin-cutaneous and melanoma (SKCM) and non-small lung cancer (NSCLC) cohorts. Crucially, even though checkpoint response predictive biomarkers are not necessarily survival prognostic, substantial evidence has already accumulated about the positive survival effects of cytolytic immune activity in melanoma and lung cancer6,7. These findings strongly suggest that immune selection pressure significantly modulates treatment naive tumor fitness. Our central hypothesis was that the predictive tumor fitness model of response to checkpoint inhibitors would be prognostic for survival in cohorts where cytolytic activity imparts a significant survival benefit. Indeed, the tumor fitness model makes no mathematical distinction between endogenous or checkpoint-induced immune response4.
An important corollary to our hypothesis was to directly test a key, central assumption of the Luksza et al. model, namely that viral similarity drives differential immune recognition of neoepitopes4. We hypothesized that in the setting of virally infected tumors, with the simultaneous presence of viral epitopes and neoepitopes, we could directly determine if the predicted MHC-I binding affinity (putative immunogenicity) of viral peptides was higher, lower, or indistinguishable from that of tumor neoepitopes from the same patients. Developing and deploying new in-silico methods, we carried this test out using the hepatitis B virally positive portion (HBV+) of the liver hepatocellular carcinoma (LIHC) TCGA liver cancer cohort.
In this brief report we found no significant correlation between tumor fitness and immune response signatures, nor was patient survival predicted using tumor fitness better than with ordinary tumor mutation burden (TMB), in either SKCM or NSCLC cohorts. Indeed we show that, as demonstrated in the checkpoint treatment setting8, a simple expression signature capturing tumor-infiltrating T-cell activity (CYT) significantly outperformed TMB in predicting survival in SKCM, and nearly so in NSCLC. We also show that tumor neoepitopes actually dominated viral epitopes in average binding affinity modulated immunogenicity within the same patients, and also better correlated with immune response, in the HBV+ HCC LIHC cohort.
Results
CYT, not Tumor fitness, correlates with immune measures in NSCLC and SKCM
We selected 305 patients with non-small cell lung cancer (NSCLC) and 337 with skin cutaneous and melanoma (SKCM) in the TCGA with matched whole exome sequencing (WES) and RNA-seq data (Fig. 1B). Putative tumor neoepitopes (ic50 < 500 nM) with significant homology with viral sequences (e-value < 10) from the IEDB (see Methods) were found in 204/337 (60%) of SKCM and 221/306 (72%) of NSCLC patients. We found that SKCM patients with viral-like epitopes had a significantly worse survival rate, and correlated with lower TMB (but not CYT), while no survival associations were observed for NSCLC (Supp. Fig. 3). We observed significant associations between CYT and TIL burden for both the NSCLC and SKCM cohorts (Fig. 2A,B), as previously observed in checkpoint treated matched cohorts8. Significantly, however, we found that maximal clonal immunogenicity Ϊ does not correlate with TIL burden or CYT in either cohort.
Tumor immunogenicity, cytolytic immune activity, and patient survival
Consistent with previous reports of the strong predictive power of immune cytolytic activity in checkpoint treated patients2,8, we found that the gene expression signature of CYT was significantly associated with survival in the non-checkpoint treated patients in SKCM and NSCLC (Fig. 3B,D). In order to assess if tumor fitness predicted survival in these cohorts, we computed maximal clonal tumor fitness scores in each patient (using the same method, software versions, and parameter specifications as Luksza et al.; see Methods) and found that they did not separate overall survival in either SKCM or NSCLC cohorts (Fig. 3A,C). As observed in other studies9, it is challenging for neoepitope based techniques to predict patient survival better than TMB alone, particularly when regressing out powerful covariates such as clinical tumor stage. To establish the relative predictive power of tumor fitness, we constructed a series of multivariable Cox proportional hazard models for each cohort which included TMB, tumor stage, Ϊ, and CYT. For each model, we computed the time dependent Brier score (see Methods; briefly, the weighted average of the squared distances between the observed survival status and the predicted survival probability of a model) and compared the resulting prediction error curves. We compared the relative predictive power of different regression models via bootstrap resampling. As shown in Fig. 3, tumor fitness does not significantly reduce the prediction error in NSCLC nor SKCM compared to models with TMB alone, while CYT does so in SKCM but not in NSCLC (Fig. 3E,F). Computing the integrated Brier scores under cross-validation confirms these results are robust.
Neoepitopes, not viral epitopes, better bind MHC-I and also associate with immune response in HBV-related liver cancer
To examine the relationship between TCR recognition and neoepitope sequence similarity to pathogen derived epitopes in samples that coexpress both, we examined patients with hepatocellular carcinoma co-infected with HBV from the LIHC TCGA dataset. We assessed 190 patients with matched WES and RNA-Seq data, detecting HBV transcripts in 115/190 (60%) via HBV genome assembly from RNA-seq data unmapped to human genes (see Methods). Of these, 88 patients produced HBV matching protein sequences through ORFfinder. Using RNA-seq data for patient HLA typing, we estimated the distribution of binding affinities of the viral epitopes and compared it with that of neoepitopes produced by tumor mutations. A total of 133,834 unique tumor neoepitopes and 30,357 HBV viral epitopes were predicted across the patients. Cumulative distribution functions of the predicted MHC-I binding affinities indicate a significant MHC-I binding affinity preference for tumor derived neoepitopes over that of HBV viral cofactors (Fig. 4). This result is robust against subsampling among neoepitopes (Supp. Fig. 1). In addition, we found that tumor neoepitope burden is modestly but significantly associated with patient TIL burden (Spearman’s ρ = 0.24, p = 0.038), but HBV epitope burden was not (restricting to MHC-I strong binders, ic50 < 200 nM; Supp. Fig. 2), further suggesting that TIL recruitment is preferentially driven by relative neoepitope burden compared to viral burden. Our observation forms a counterexample to the model assumption that viral sequence similarity drives preferential epitope recognition.
Discussion
Determining the degree of ‘non-selfness’ of mutant peptides in the face of enormous individual variability of TCR repertoires is notoriously difficult, and yet critical, to answer at scale. Point somatic mutations located outside the MHC-binding anchor positions may appear to be more ‘non-self’ since they may lead to significantly increased peptide-MHC binding compared with wildtype sequence. However it has traditionally not been straightforward to translate this notion into a better survival or treatment response predictor. The relative spectra of subclonal and clonal immunogenic mutations, and similarly mutant sequence homology with experimentally validated pathogen epitopes, potentially contain important tumor evolution signatures which may indicate intriguing similarities with pathogen evolution.
To gain further intuition into a recent neoepitope based tumor fitness scoring model4, we applied it to checkpoint-treatment-naive samples of the same tumor types in the TCGA. A key aim was to test how this model predicts survival given that it has shown promise in predicting response to immune checkpoint inhibitors. Although this setting may involve substantially different (thus far uncharacterized) tumor-immune dynamics, we reasoned that a sufficiently general notion of tumor fitness should inversely correlate with patient survival, especially in those cohorts where TIL and CYT are already known as strong survival correlates.
Our results suggest that the proposed tumor fitness measure based on pathogen epitope similarity did not correlate with CYT in either TCGA NSCLC nor SKCM, potentially indicating that it is specific to the patients treated with checkpoint inhibitors. Furthermore, in a test of a key model assumption, in the case of HBV-related HCC in the TCCA LIHC, we found that even when viral epitopes are competing with neoepitopes for T-cell presentation there was actually a substantial average bias towards higher MHC-I binding affinity for the neoepitopes. In addition, the significant correlation between TMB and TIL burden further suggests that the tumor dominates immune recruitment over HBV viral epitopes. It is interesting to speculate on whether other tumors with known viral cofactors, such as the Human Papillomavirus in cervical or head and neck cancers, and Epstein Barr virus in Burkitt’s lymphoma, show similar biases.
Furthermore, we found that tumor fitness in the treatment-naive samples does not improve prediction error of survival beyond that of TMB and clinical tumor staging alone, while a simple measure of CYT did so in SKCM. The tumor fitness model has two key degrees of freedom, ɑ and k, that were specifically tuned to reduce prediction error in patients treated with checkpoint inhibitors4. However, given that the alignment score distributions for the TCGA cohorts were not significantly different from the matched checkpoint treated settings (Supp. Fig. 4), we used the same values in our modeling of these treatment-naive samples. While these data suggest that the tumor immune dynamics in treated patients may be notably different than in untreated patients, we suspect it may also simply reflect a strong dataset-specific and parameter-tuning dependence of the tumor fitness score. Indeed, as noted the CYT measure alone seemed to reduce survival prediction error, as found by others10,11 and Luksza et al. themselves as a useful component to add to the tumor fitness score. Though the improvement of survival prediction by the inclusion of CYT may be, essentially, expected based on its efficacy as an independent predictor of therapy responsiveness in the checkpoint-setting8, we demonstrated here that this common-sense expectation holds true in the non-checkpoint setting as well.
Although our study points out key limitations in extending the paradigm of tumor fitness scoring to untreated patients, it does have the same caveats any in-silico immunogenicity estimation based analysis method has. Indeed, the accuracy of in-silico predictions of neoepitopes or viral binding affinity in estimating immune reactivity are at best suboptimal compared to using mass spectroscopy. Also, in comparing viral and neoepitopes it is important to note that several lines of evidence suggest that the replication status of HBV is dependent on the differentiation status of the hepatocytes12,13. Thus, the virus may hardly replicate in HCC hepatocytes and further reduce the viral epitope burden within the tumor.
Overall our study indicates that while tumor fitness scores can be used to predict response to checkpoint inhibitors, they do not seem to predict survival in untreated patients where endogenous immune activation is a significant survival factor. It is important to note that while this tumor fitness measure and its underlying assumptions have been adapted with positive predictive results in other contexts14,15, we found that the exact form of the model was not predictive in these endogenous cases. Nevertheless, it is encouraging to observe that CYT measures are powerful in both treated and untreated patients, which may point to the existence of universally predictive and prognostic immune activation signatures.
Methods
TCGA analysis
RNA-Seq data for the TCGA patient cohorts was aligned to Hg38 with STAR (v2.5.1b) in two-pass mode16. Gene counts for Gencode v23 (www.gencodegenes.org) gene annotations were generated using featureCounts. Read counts underwent TMM normalization and logCPM transformation using voom17. Mapped reads were used to allelotype (MHC class-I loci) each patient18, and estimate the putative TIL burden and clonality per patient by profiling TCR and BCR sequences with MiXCR19, normalizing by patient library size. To predict neoepitopes and associated viral epitope burden, we used Topiary20 to call mutation-derived cancer T-cell epitopes from somatic variants, tumor RNA expression data, and patient class-I HLA type. This tool matches mutations with gene annotations, filters out non-protein coding changes, and finally creates a window around amino acid changes, which is then fed into NetMHCCons for each patient HLA allele across tiles of 9–12 aa in length. Given that HLA-I processes neoepitopes by degradation to non-conformational 8–11 amino-acid residues, we excluded neoepitopes with mutations obscured to T-cells within HLA-I binding pockets (anchor residues at 2,9 aa that are mutated with respect to the wild-type form of the epitope). In the case of frameshift mutations, in principle this window starts from the mutation minus the length of the peptide up to the first stop codon. Neoepitopes were filtered for noise using a binding affinity threshold of ic50 < 10,000 nM, which removed 89% of predictions. A measure of tumor mutation burden (TMB) for each patient was calculated by enumerating the number of called DNA-mutations in patient associated MAF files.
Tumor fitness score
We assigned tumor immunogenicity scores following the method outlined by Luksza et al. Briefly, a protein BLAST of the neoepitopes from the SKCM and NSCLC cohorts was performed per patient against a database of viral epitopes sourced from the IEDB5; the best hits with higher sequence similarity normalized by sequence database size (bitscore) for each neoepitope were retained. A second, randomized set of neoepitopes was also blasted, as a negative control group. Smith-Waterman alignments scores (|s, e|) were calculated for each match, following a strict 500 nM binding affinity threshold filtering for all predicted neoepitopes. For each patient, a TCR recognition potential (R) was calculated by modeling the probability for a given neoepitope to be bound to a viral match from the IEDB using:
1 |
with constants ɑ (binding curve displacement) and k (curve steepness) retained from the original model in matching to each TCGA patient cohort, after comparisons of alignment score distributions with checkpoint treated cohorts revealed no significant differences (Supp. Fig. 4). For each neoepitope, we calculated an MHC-I binding affinity amplitude score (A), as a ratio of the binding affinities of the mutant neoepitope and the wild type epitope, which were corrected for dissociation constant bias by multiplying by the reciprocal max among all wild type epitopes. Tumor immunogenicity was assigned by selecting the maximum product of these scores from among all neoepitopes across patient clones, j:
2 |
This term represents the effective expected size of immune response and is used to rank the dominant immunogenic clone per patient. The inverse of this score, tumor fitness, describes the likelihood of evading immune detection.
Cytolytic T-Cell activity (CYT) scores were calculated using the geometric mean of the RPKM expression of granzyme A (GZMA) and perforin-1 (PRF1)6.
HBV antigen calling
Raw RNA sequencing reads that did not map to the GRCh38 reference genome was assembled into contigs using Trinity (using –no_run_chrysalis –no_run_butterfly flags, which only invokes the Inchworm step) to perform greedy kmer-25 contig assembly21. Resulting contigs with a sufficiently high entropy (to exclude homopolymer sequences), at least 100 bp long and supported by at least 20 reads were retained and aligned with BLAST against the HBVdb22 reference genomes (e-value < 1e-6) for HBV (genotypes A, B, C, D, E, F, and G), effectively identifying the HBV positive patients (n = 115). CD-Hit was used to consolidate highly similar contig sequences (e-values < 1e-10)23,24. Finally, ORFfinder was used to predict the putative HBV protein products from each contig, which were validated and filtered (e-value < 1e-10) by a protein BLAST against an HBV protein database compiled from HBVdb for the HBV genotypes above25. Importantly, our contig length filtering means that ORFs shorter than 300 aa are excised. HBV epitopes were predicted using the matching protein products and patient HLA-typing calls with Topiary (see above), and predicted peptides were filtered for noise by binding affinity (ic50 < 10,000 nM), reducing predictions by 94%.
Statistical analysis
Significance for all test measures were determined at p-values < 0.05. Spearman correlations between expression based tumor immune measures (CYT, TIL burden, TIL clonality) and TMB and immunogenicity score (Ϊ) were calculated to correct for outliers. We used a one-sided Kolmogorov-Smirnov test to compare viral and neoepitope MHC-I binding affinity distributions, with an alternative hypothesis that the neoepitope distribution falls above the viral distribution. Additionally, to assure that the MHC-I binding affinity bias toward tumor neoepitopes was not confounded by a larger epitope set, we subsampled 10,000 epitopes from both populations and compared binding affinity distributions by one-sided (neoepitope greater) Kolmogorov-Smirnov test for 1,000 replicates (Supp. Fig. 1).
Kaplan-Meier survival curves and risk tables using patient tumor immunogenicity, Ϊ, and immune activity, CYT, were created with the survival and survminer packages26,27. Patient grouping into low and high tumor immunogenicity and immune activity was done using the bottom and top 15% quantiles for these measures, respectively. Cox proportional hazard models using tumor stage and TMB, Ϊ, and CYT covariates were constructed using the rms package28. To address possible violation of the assumption of proportional hazard ratios between populations in our survival models, we computed the correlation between survival covariates (Ϊ, CYT) and time for each model. We found all model covariates were time-independent, and thus a log-rank test was used to evaluate significance in predicted survival outcomes between groups for Kaplan-Meier survival estimates. The relative predictive power of various Cox survival models was evaluated using the pec R package29 by computing the Integrated Brier Score (IBS) across 0.632+ bootstrap resampling with 100 iterations (B = 100). We used the Kaplan-Meier estimator for the censoring times in the bootstrap simulations. The weights in the IBS correspond to the probability of not being censored. Model error curves were compared using Wilcoxon test for IBS comprising approximately 80% of total patients, with test significance provided in graphic tables (Fig. 3E,F). Error prediction reduction measured by IBS with respect to time from diagnosis (5 years) was also evaluated (Supp. Fig. 5). IBS cross validation was performed by sampling Brier scores with 100 iterations across random patient groups for each model and applying a one-sided (TMB_I less) Kolmogorov-Smirnov test.
Supplementary information
Author contributions
B.L. conceived and directed research. N.A. and A.B. performed principal data analysis. A.B., A.V., E.G.K. and B.L. wrote and edited the manuscript.
Data and Code availability
All code and non-protected data is available upon reasonable request to the corresponding author.
Competing interests
The authors declare no competing interests.
Footnotes
Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Supplementary information
is available for this paper at 10.1038/s41598-020-61992-2.
References
- 1.Sharma P, Hu-Lieskovan S, Wargo JA, Ribas A. Primary, Adaptive, and Acquired Resistance to Cancer Immunotherapy. Cell. 2017;168:707–723. doi: 10.1016/j.cell.2017.01.017. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.Riaz N, et al. Tumor and Microenvironment Evolution during Immunotherapy with Nivolumab. Cell. 2017;171:934–949.e16. doi: 10.1016/j.cell.2017.09.028. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.Rizvi NA, et al. Cancer immunology. Mutational landscape determines sensitivity to PD-1 blockade in non-small cell lung cancer. Science. 2015;348:124–128. doi: 10.1126/science.aaa1348. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Łuksza M, et al. A neoantigen fitness model predicts tumour response to checkpoint blockade immunotherapy. Nature. 2017;551:517–520. doi: 10.1038/nature24473. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Vita R, et al. The Immune Epitope Database (IEDB): 2018 update. Nucleic Acids Res. 2019;47:D339–D343. doi: 10.1093/nar/gky1006. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Rooney MS, Shukla SA, Wu CJ, Getz G, Hacohen N. Molecular and genetic properties of tumors associated with local immune cytolytic activity. Cell. 2015;160:48–61. doi: 10.1016/j.cell.2014.12.033. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Barnes TA, Amir E. HYPE or HOPE: the prognostic value of infiltrating immune cells in cancer. British Journal of Cancer. 2017;117:451–460. doi: 10.1038/bjc.2017.220. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Cristescu, R. et al. Pan-tumor genomic biomarkers for PD-1 checkpoint blockade-based immunotherapy. Science362 (2018). [DOI] [PMC free article] [PubMed]
- 9.Havel JJ, Chowell D, Chan TA. The evolving landscape of biomarkers for checkpoint inhibitor immunotherapy. Nat. Rev. Cancer. 2019;19:133–150. doi: 10.1038/s41568-019-0116-x. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Wakiyama H, et al. Cytolytic Activity (CYT) Score Is a Prognostic Biomarker Reflecting Host Immune Status in Hepatocellular Carcinoma (HCC) Anticancer Res. 2018;38:6631–6638. doi: 10.21873/anticanres.13030. [DOI] [PubMed] [Google Scholar]
- 11.Roufas C, et al. The Expression and Prognostic Impact of Immune Cytolytic Activity-Related Markers in Human Malignancies: A Comprehensive Meta-analysis. Front. Oncol. 2018;8:27. doi: 10.3389/fonc.2018.00027. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Halgand B, et al. Hepatitis B Virus Pregenomic RNA in Hepatocellular Carcinoma: A Nosological and Prognostic Determinant. Hepatology. 2018;67:86–96. doi: 10.1002/hep.29463. [DOI] [PubMed] [Google Scholar]
- 13.Fu S, et al. Detection of HBV DNA and antigens in HBsAg-positive patients with primary hepatocellular carcinoma. Clin. Res. Hepatol. Gastroenterol. 2017;41:415–423. doi: 10.1016/j.clinre.2017.01.009. [DOI] [PubMed] [Google Scholar]
- 14.Zhang, J. et al. The combination of neoantigen quality and T lymphocyte infiltrates identifies glioblastomas with the longest survival. Communications Biology vol. 2 (2019). [DOI] [PMC free article] [PubMed]
- 15.Wood MA, et al. Population-level distribution and putative immunogenicity of cancer neoepitopes. BMC Cancer. 2018;18:414. doi: 10.1186/s12885-018-4325-6. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16.Dobin A, et al. STAR: ultrafast universal RNA-seq aligner. Bioinformatics. 2013;29:15–21. doi: 10.1093/bioinformatics/bts635. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17.Ritchie ME, et al. limma powers differential expression analyses for RNA-sequencing and microarray studies. Nucleic Acids Research. 2015;43:e47–e47. doi: 10.1093/nar/gkv007. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18.Szolek A. HLA Typing from Short-Read Sequencing Data with OptiType. Methods Mol. Biol. 2018;1802:215–223. doi: 10.1007/978-1-4939-8546-3_15. [DOI] [PubMed] [Google Scholar]
- 19.Bolotin DA, et al. MiXCR: software for comprehensive adaptive immunity profiling. Nat. Methods. 2015;12:380–381. doi: 10.1038/nmeth.3364. [DOI] [PubMed] [Google Scholar]
- 20.Rubinsteyn, A. & Nathanson, T. Topiary: predict mutated T-cell epitopes from sequencing data. Github repository, https://github.com/openvax/topiary (2018).
- 21.Haas BJ, et al. De novo transcript sequence reconstruction from RNA-seq using the Trinity platform for reference generation and analysis. Nat. Protoc. 2013;8:1494–1512. doi: 10.1038/nprot.2013.084. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22.Hayer J, et al. HBVdb: a knowledge database for Hepatitis B Virus. Nucleic Acids Res. 2013;41:D566–70. doi: 10.1093/nar/gks1022. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 23.Fu L, Niu B, Zhu Z, Wu S, Li W. CD-HIT: accelerated for clustering the next-generation sequencing data. Bioinformatics. 2012;28:3150–3152. doi: 10.1093/bioinformatics/bts565. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 24.Li W, Godzik A. Cd-hit: a fast program for clustering and comparing large sets of protein or nucleotide sequences. Bioinformatics. 2006;22:1658–1659. doi: 10.1093/bioinformatics/btl158. [DOI] [PubMed] [Google Scholar]
- 25.Rombel IT, Sykes KF, Rayner S, Johnston SA. ORF-FINDER: a vector for high-throughput gene identification. Gene. 2002;282:33–41. doi: 10.1016/S0378-1119(01)00819-8. [DOI] [PubMed] [Google Scholar]
- 26.Therneau, T. A package for survival analysis in S. R package version 2.4.4, https://CRAN.R-project.org/package=survival (2019).
- 27.Kassambara, A. & Kosinski, M. Survminer: drawing survival curves using ggplot2. R package version 0.4.4, https://CRAN.R-project.org/package=survminer (2019).
- 28.Harrel, F. E. Jr. rms: regression modeling strategies. R package version 5.1, https://CRAN.R-project.org/package=rms (2019).
- 29.Mogensen UB, Ishwaran H, Gerds TA. Evaluating Random Forests for Survival Analysis using Prediction Error Curves. J. Stat. Softw. 2012;50:1–23. doi: 10.18637/jss.v050.i11. [DOI] [PMC free article] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Supplementary Materials
Data Availability Statement
All code and non-protected data is available upon reasonable request to the corresponding author.