Devising a Consensus Framework for Validation of Novel Human Coding Loci

Abstract

A report on the Wellcome Trust retreat on devising a consensus framework for the validation of novel human protein coding loci, held in Hinxton, U.K., May 11–13, 2015.

INTRODUCTION

At the HUPO 2011 Congress the Chromosome-Centric Human Proteome Project (c-HPP) was officially launched as a large multidisciplinary international effort to perform the cataloguing and characterization of human gene products with a focus on loci that have little or no evidence at the protein level, termed “missing proteins”.¹ At this time there was no satisfactory evidence at the protein level for 33% of the 20 059 protein coding genes. Since this initial assessment the number of missing proteins has been reduced considerably, but as of October 2014, ~18% of the human proteome was still lacking evidence at the protein level.² Different reasons can explain why some proteins are difficult to identify by mass spectrometry (MS) or antibody-based strategies. Some of them may be expressed in rarely studied tissues or cell types or are only expressed/induced in very specific conditions. Others are expressed at levels that are below the detection limits of the current MS instruments. In addition, some protein sequences do not contain tryptic peptides or epitopes that can unambiguously identify the protein. Finally, a number of protein-coding loci that were predicted from genomic analysis simply do not exist, encode pseudogenes, or give rise to nonprotein coding RNA. With technological advances in the proteogenomics field and better access to rare samples, solutions to overcome the issues of rare or low abundance proteins are emerging; however, due to increased data set sizes and lower detection limits, the fraction of incorrect protein identifications tends to increase. Furthermore, a number of recent publications have claimed to have found “new” proteins overlooked by the main reference databases, but careful reanalysis of the evidence provided indicates that many of these claims are invalid.

The “Devising a Consensus Framework for Validation of Novel Human Coding Loci” workshop was held between 11th and 13th May, 2015, at the Wellcome Genome Campus at Hinxton, bringing together invited experts from the fields of genomics, transcriptomics, MS-based proteomics, and representatives from the major human genome and protein databases and proteomic repositories. A test set of putative protein coding loci from recent publications was selected and distributed to the participating annotation resources (HAVA-NA,³ neXtProt,⁴ RefSeq,⁵ and Swiss-Prot⁶) prior to the workshop by the organizing committee (Jen Harrow (GENCODE⁷), Elspeth Bruford (HGNC⁸), and Lydie Lane and Amos Bairoch (neXtProt⁴)). The central focus of the meeting was discussing these edge cases and consolidating a list of putative human protein products, with the eventual aim of formulating guidelines for publication that the biocuration and scientific research community can use to determine the evidence needed to support a novel locus being included or excluded from the reference human protein-coding gene set.

IDENTIFYING/CONFIRMING NOVEL PROTEINS USING MASS SPEC AND RIBOSOME PROFILING

This session began with presentations from Alexey Nesvizhskii from University of Michigan (Ann Arbor, MI), Michael Tress from the Spanish National Cancer Research Centre (CNIO; Madrid, Spain), and Eric Deutsch from the Institute for Systems Biology (Seattle, WA). Nesvizhskii discussed false discovery rates (FDRs) in MS-based data sets, which are statistical measures of the accuracy of the protein identification. He stressed how the FDR cut-offs must be adjusted depending on the locus types being studied. He also introduced his Web site CRAPome,⁹ the Contaminant Repository of Affinity Purification Data, which aggregates background contaminants from negative controls of affinity purification-MS studies. Tress, who works with the GENCODE consortium, also stressed the importance of FDRs and suggested that the participants of the workshop should devise some rules for filtering proteomic data to ensure that only high confidence data is used for validation. Two recent large-scale proteomics studies reported a large number of “novel protein coding genes”;^10,11 however, the analysis of the GENCODE consortium¹² showed that these two studies reported dubious peptide evidence for at least 200 olfactory receptor genes (which encode transmembrane proteins with highly restricted expression), even though the studies had not included data from olfactory tissue. Tress cited three possible reasons for the high number of erroneous predictions: Some of the identified peptides match to more than one gene; some peptides were wrongly identified as containing a glutamine to pyroglutamic acid modification in non N-terminal positions; and both studies included many low quality spectra. Deutsch, who is in charge of the PeptideAtlas,¹³ warned the participants of the danger of combining data sets as you can end up with a very high FDR rate; he said that the thresholds used in PeptideAtlas are very stringent to avoid random noise. For example, PeptideAtlas identifies only two discriminating olfactory receptor peptides, and even these two have now been reassessed as having more likely matches to peptide variants of abundant proteins. Evidence is regarded as insufficient if the peptides are fewer than seven residues or weak if fewer than two different (but potentially overlapping) peptides support one protein. In their 2105-03 release, PeptideAtlas has now raised the bar to 9 peptides of 9 or more amino acids.¹³ Gil Omenn of the University of Michigan (Ann Arbor, MI), chair of the HPP², commented that structural proteomics can also be useful in validation by predicting if the protein, including splice isoforms, may be functional. This was followed by discussion of some of the test set of loci that had been distributed prior to the meeting. The key point arising from this discussion was that the different annotation resources (HAVANA, neXtProt, RefSeq, and Swiss-Prot) were generally in agreement about their criteria for deciding coding status (e.g., evolutionary conservation, transcript evidence, ribosome profiling, published experimental characterization) but required help to decide whether MS data for a potential new ORF were of high enough quality to trigger further analysis. To this end, Deutsch suggested that candidate new proteins be sent to PeptideAtlas and that twice a year they would be processed against the combined set of human MS spectra and valid peptides reported back. Nesvizhskii proposed giving access to a web-based tool that will summarize the reliability of a peptide identification.

IDENTIFYING SMALL ORFs AND uORFs

Jon Mudge, from the HAVANA group (Hinxton, U.K.), started this session by describing their annotation of small coding DNA sequences (CDSs). This led to a discussion of the term “uORF” (upstream open reading frame), as some felt this may ascribe a function to the locus of regulating the translation of the CDS. Mudge stated that most 5' untranslated regions (UTRs) contain some kind of uORF, so should they all be annotated, or only ones where some functional data has been found? Juan Pablo Couso from the University of Sussex highlighted that there is a continuum between clearly defined protein coding loci and characterized nonprotein coding loci. Translation can be inferred from the results of ribosomal profiling that assays the regions of mRNA protected by ribosomes while undergoing translation. A combination of ribosomal profiling and MS data has been used to identify small ORFs (smORFs) that are common in Drosophila¹⁴ and could potentially be emergent protein coding genes. Conservation across species can also argue in favor of the existence of these small proteins; however, Couso emphasized the difficulty in using BLAST for identifying smORFs under 70 amino acids (aa). It was proposed that smORFs of >70 aa be classed as protein coding if there were at least two lines of evidence (e.g., conservation and ribosome profiling), but smORFs of <70 aa would only be regarded as protein coding if experimental evidence at the protein level was available; however, it should be borne in mind that detection of such small proteins by MS can be challenging, as they are usually low abundance, have only a few tryptic sites, and are spatially/temporally restricted. From the few examples discussed at the end of the session, it appeared that it was sometimes difficult for curators to decide whether the data should be represented as a one-gene or two-gene model.

PSEUDOGENES CONFUSING THE PROTEOME LANDSCAPE

The next workshop session focused on pseudogenes and began with Cristina Sisu from Yale University (New Haven, CT), who presented her analysis of the human, worm, and fly pseudogenes,¹⁵ highlighting their finding of great variation in pseudogene composition between the species. Of particular relevance to the meeting were their criteria for predicting transcribed and translated pseudogenes in human. Sisu highlighted that there is currently little overlap between the 1098 transcribed pseudogenes predicted by the HAVANA group and the 1441 predicted automatically using RNA-seq data mapping to GENCODE 19. Harsha Gowda from the Institute of Bioinformatics in Bangalore (India) discussed transcribed and translated human pseudogenes from the Pandey lab,¹¹ although it was evident that their analysis was less stringent. Jyoti Choudhary from the WTSI (Hinxton, U.K.) highlighted their validation of translated pseudogenes from analysis in mouse;¹⁶ however, the number of identified and validated loci is extremely low (around nine). Finally, Kim Pruitt from NCBI highlighted issues of annotating pseudogenes and correctly mapping expression data between pseudogenes and the parent gene from which they are derived. RefSeq has currently predicted over 1032 transcribed pseudogenes. If a pseudogene has coding potential, then RefSeq curators will change it to a protein coding gene and it will no longer be tracked as pseudogene, unlike HAVANA, who annotate a pseudogene with coding potential or an MS hit as a translated pseudogene. At the end of the session, the annotation groups discussed difficult annotation examples, where assignment as a pseudogene or protein coding gene was ambiguous and involved investigating variation of haplotypes, underlying genome assembly errors and using extremely stringent mapping options to confidentially change predictions to coding genes.

LONG NONCODING RNAS

The workshop focus then switched to long noncoding RNAs (lncRNAs) with remote presentations from Chris Ponting from the University of Oxford (Oxford, U.K.), Mitch Guttman from Stanford University (Stanford, CA), and Tim Mercer from the Garvan Institute of Medical Research (Sydney, Australia) as well as Peter Jan Volders and Kenneth Verheggen from Ghent University (Gent, Belgium). Ponting discussed the low degree of conservation of most lncRNAs,¹⁷ including the example of the RNA gene H19, which contains ORFs that are not conserved and hence unlikely to encode proteins. He also mentioned that conversely there are lineage-specific examples of protein coding genes that are not well-conserved, such as the 2 Mb EYS gene that is mutated in some cases of retinitis pigmentosa in humans but has been pseudogenised in rodents. Guttman likewise mentioned the importance of evolutionary conservation; for example, as measured by a PhyloCSF score, which utilizes a multispecies genome alignment to identify conserved regions in determining coding status. He also talked about the ribosome release score (RRS),¹⁸ a ratio of the ribosome profiling reads in an ORF compared with the reads in the 3′ UTR of a transcript, which can be used to differentiate translated and nontranslated RNAs. There are very few lncRNAs that have both a high RRS and a high PhyloCSF score. Mercer talked about using CaptureSeq to identify novel lncRNA transcripts.¹⁹ While RNA-seq is good at detecting highly expressed genes, it is not so good at identifying transcripts that are more weakly expressed. In contrast, CaptureSeq is selective RNA sequencing that uses short nucleotide probes to target specific regions of interest and hence can provide improved detection and assembly of lncRNA gene models. Volders and Verheggen both represented the LNCipedia resource,²⁰ and Volders discussed the need for a gold standard for ribosome profiling data to enable comparisons between results, while Verheggen introduced their ReSpin pipeline to reprocess data from the PRIDE protein interactions database and told us how subtracting these data from transcripts with a low PhyloCSF score had enabled them to identify a high confidence set of lncRNA transcripts.

HUMAN VARIATION

This session was devoted to human variation and its impact on genome annotation, with presentations from Marie-Paule Lefranc from the Institut de Génétique Humaine (Montpelier, France), Jens Mayer from the University of Saarland (Saarbrücken, Germany), and a remote presentation from Tsviya Olender at the Weizmann Institute (Rehovot, Israel). Lefranc introduced IMGT,²¹ the international ImMunoGeneTics Information System, and highlighted the structural conservation of immunoglobulin variable domains combined with their very high amino acid diversity; therefore, there can be huge variability in the resulting peptides, and some peptides may be expressed very specifically or at extremely low levels, making their detection very difficult. Mayer discussed transposable elements, in particular, endogenous retroviruses (ERVs) and long interspersed nuclear elements (LINEs), in the human genome. ERVs and related sequences comprise ~8% of the human genome, but only the ERV-K family has continued to replicate in the human genome and the loci are highly polymorphic but also highly similar,²² making it difficult to distinguish which loci are actually protein coding. Olender, from the HORDE database of olfactory receptors,²³ talked about their recent analysis of the human olfactory receptor repertoire (the largest gene family in the human genome) using Next Generation Sequencing (NGS) data from 4 samples of human olfactory epithelium versus 16 control tissues. Clonal expression of OR transcripts in individual olfactory epithelial cells, might make detection of transcripts, let alone proteins, more problematic. As well as being highly restricted in their expression, some olfactory receptor loci can vary in coding status between individual genomes (called polymorphic or segregating pseudogenes), so they have also devised a metric called the “CORP” (Classifier for Olfactory Receptor Pseudogenes) score to estimate pseudogene probability of a given allele.

CONCLUSIONS AND NEXT STEPS

It was clear from the workshop that there is a lot of ambiguity surrounding how the different annotation groups interpret proteomics data when using it as evidence for a novel protein coding gene. The groups were reticent to use proteomic data alone for validation, unless there were multiple high confidence hits to support it. The curation teams wanted to improve communication between the different groups (GENCODE, HGNC, neXtProt, RefSeq, Swiss-Prot) when trying to evaluate these edge cases as putative novel protein coding genes and proposed using the existing consensus CDS (CCDS) system²⁴ provided by the NCBI to record the exchanges between curators. We hope that further examination of edge cases will be used in the development of guidelines from the curation groups that will aid the correct interpretation of MS evidence to validate novel protein coding genes. Members of the proteomics community attending the workshop are continuing discussions concerning the development of tools that the annotation community could use to distinguish between highquality MS data and low-quality data that should not be used for evidence. PeptideAtlas (Deutsch) agreed to consider incorporating RefSeq as an additional reference proteome, probably in the next annual update. The annotation groups also proposed providing novel RNA-seq models for the proteogenomics community to increase their search space and hence the opportunities to find novel proteins. Conversely, it was recommended that claimed peptide matches to novel proteins from pseudogenes or lncRNAs should be searched for highquality matches to known sequences with single amino acid variants or isobaric post-translational modifications. Novel annotations from chromosomes 14 and 21 will be used for a pilot analysis to see if MS data can be used in combination with other data to identify novel protein coding genes. Finally, the participants felt it would be useful to rerun the workshop at a suitable frequency (1 to 2 years) to discuss updates in technology and how the different annotation groups can incorporate these changes into their analyses.