Challenges and Opportunities for Single-cell Computational Proteomics

Abstract

Single-cell proteomics is growing rapidly and has made several technological advancements. As most research has been focused on improving instrumentation and sample preparation methods, very little attention has been given to algorithms responsible for identifying and quantifying proteins. Given the inherent difference between bulk data and single-cell data, it is necessary to realize that current algorithms being employed on single-cell data were designed for bulk data and have underlying assumptions that may not hold true for single-cell data. In order to develop and optimize algorithms for single-cell data, we need to characterize the differences between single-cell data and bulk data and assess how current algorithms perform on single-cell data. Here, we present a review of algorithms responsible for identifying and quantifying peptides and proteins. We will give a review of how each type of algorithm works, assumptions it relies on, how it performs on single-cell data, and possible optimizations and solutions that could be used to address the differences in single-cell data.

Graphical Abstract

Highlights

• •Single cell proteomics (SCP) data is rapidly becoming available.
• •Proteomics workflows and algorithms were optimized for bulk datasets.
• •The low signal of SCP data challenges assumptions of computational workflows.
• •Tradeoff between identification and quantification in experimental designs.
• •Uncertainty in protein quant from high percentage of missing peptide abundances.

In Brief

Single-cell proteomics is growing rapidly, but most research has focused on improving instrumentation and sample preparation. Current algorithms being employed on single-cell data were designed for bulk data and have underlying assumptions that may not hold true for single-cell data. Here, we present a review of algorithms for identifying and quantifying peptides and proteins, including how each type of algorithm works, assumptions it relies on, and possible optimizations for single-cell data.

Complex biological systems consist of numerous individual cells sensing and responding to their environment. Most systems include a variety of heterogeneous cells and therefore our understanding of the system compels us to know these individual and diverse cells through single-cell measurements. Quantifying the proteins within individual cells is essential to characterize the cell, since proteins perform most cellular functions. Mass spectrometry–based proteomics enables high throughput profiling of proteins within a sample and has proved to be valuable in cellular biology and biomedical research (1, 2, 3, 4, 5). Traditional proteomics workflows analyze all cells in a sample together and therefore measure population averages. This not only confounds differences that exist between individual cells but also makes it difficult to detect cell-type–specific proteins which are low-abundance in the bulk homogenate (6, 7). To address these problems, researchers are rapidly improving the capabilities for single-cell proteomics (SCP).

Current progress in SCP has been heavily focused on developing methods for sample preparation, chromatography, and data acquisition. These advancements have been previously reviewed (8, 9, 10). It is important to understand that changes to data acquisition parameters can affect algorithm performance in unexpected ways—specifically, they may improve data quality for peptide identification but detract from quantitative accuracy. This tradeoff is described in detail below, but a motivating example here will suffice. Various methods have been used to increase the number of ions collected for an MS2 spectrum. In a label-free experiment, the chromatography may be shortened and/or the ion acquisition time increased, thus delivering more peptide ions for fragmentation which improves the likelihood of achieving a confident peptide spectrum match (PSM). However, these actions also significantly reduce the number of MS1 spectra acquired during a peptide’s elution and could potentially harm quantitative accuracy. Thus, as experimental researchers are varying data acquisition parameters, they should be aware of how this might affect algorithms for both identification and quantification.

After significant progress in overcoming major technical hurdles for generating SCP data, attention is now turning to the algorithms used to identify and quantify proteins (11). Although a diverse set of data acquisition strategies are used in SCP, data analysis for all of these methodologies generally involves four steps: 1) peptide identification, 2) peptide quantitation, 3) protein identification, and 4) protein quantitation (12, 13). As no dedicated software yet exists for SCP analysis, researchers are currently employing algorithms that were developed for bulk data. However, single-cell data is fundamentally different from bulk data and algorithms may need to adapt for optimal performance (14). To assist researchers in developing and optimizing algorithms for single-cell data, this review analyzes common algorithms and workflows used in each of these four fundamental steps of mass spectrometry data processing. The adaptation of downstream analysis of quantitative SCP data has also started (15, 16). However, the focus of this review is to explain important algorithmic assumptions in each of the four processing steps used in quantitative proteomics, their challenges, and potential solutions for SCP.

Due to the recent emergence of single-cell proteomics and its strong focus on the development of experimental methods, the published literature on computational methods for data analysis is very limited. Therefore, this manuscript is a hybrid review of sorts. Our goal is not to provide a review of general computational methods and software developed for proteomics, as many quality reviews already exist (17, 18, 19, 20, 21); but rather our goal is to point out where common assumptions used in software for bulk proteomics will be problematic for SCP. Thus, we review the published peer-reviewed manuscripts but also provide context for unaddressed challenges.

Peptide Identification

This review is limited to bottom-up proteomics, where proteins are digested into peptides and the peptides are analyzed in the mass spectrometer. Moreover, we are primarily concerned with the aspects of peptide identification that are problematic in SCP. Two common methods exist for data acquisition in bottom-up proteomics: data-dependent analysis (DDA) and data-independent analysis (DIA). For both DIA and DDA, the first data analysis step is to identify peptides from fragmentation spectra. The data generation and analysis algorithms are different for DDA and DIA methods, and we will address each one separately. After scoring PSMs, both DDA and DIA methods assign statistical probability to the match, often using the target/decoy model (22) or a mixture modeling approach (23). The data from SCP does not change the assumptions of these approaches and so will not be discussed here.

DDA Peptide Identification

Data-dependent acquisition identifies highly abundant ions in MS1 spectra and isolates and fragments them for measurement in an MS2 spectrum (24). The most common analysis method for this data type is a database search where algorithms identify the most likely candidate peptide for each spectrum. Database search algorithms began in the 1990s (25) and have continued to evolve with new advances for novel search methods (26, 27), improved speed (28), rigorous statistical confidence (23, 29), identification of protein modifications (30, 31), combining search engines (32), open source software (33), compatibility for multiple fragmentation methods (34, 35), and user friendly and robust tools (36). The specific implementation of scoring to judge PSMs is unique to each algorithm, however, they share common fundamental assumptions. In general, these tools generate a theoretical spectrum for each candidate peptide which is compared to the observed spectrum and scored to reflect the quality of the match (reviewed in (37)). The highest scoring peptide is assigned to the spectra. Although each software tool may use slightly different metrics, there are two main features which are fundamental and common to all: the number of annotated fragment ions is expected to be high for the correct PSM and annotated ions are expected to be more intense than background/noise ions.

In previous work, we characterized these two essential scoring metrics by comparing single-cell spectra to bulk spectra (14). Figure 1 illustrates the differences between single-cell spectra and bulk spectra. We found the number of annotated fragment ion peaks to be reduced in single-cell spectra. This result is expected, as the lower ion signal diminishes the probability of observing low intensity fragment ions. Since most algorithms rely heavily on the number matching peaks between the observed and theoretical spectrum, single-cell PSMs will get a lower score and therefore be more likely to be excluded by false-discovery rate (FDR) cutoffs. We also investigated the difference in intensity between annotated peaks and background noise. In single-cell data, we found a compressed ratio of annotated peaks to background noise (S/N ratio). This compression makes it more difficult for algorithms to distinguish true fragment ion peaks. The reduced number of ions and the compressed S/N ratio make it more challenging to confidently assign a peptide to a spectrum, resulting in fewer PSMs passing FDR cutoffs. While this work was done on label-free data, it is reasonable to expect similar results in tandem mass tag (TMT) data. In the SCoPE protocol, many cells are combined into a ‘carrier channel’ to increase the number of ions and therefore improve MS2 signal. The increase in ions improves peptide identification yet substantially affects quantitative accuracy as discussed below in the section on quantification.

Fig. 1.

Comparison of single-cell and bulk MS2. For the peptide GFAFVTFDDHDSVDK, the top spectrum is from single-cell data, and the bottom spectrum is from bulk data. The bulk spectrum has 17 annotated y-ions and single-cell only has five annotated y-ions. A similar fraction of b-ions are lost in single cell spectra. The lost peaks are mostly from medium and low intensity peaks.

DIA Peptide Identification

In DIA, an MS2 spectrum is acquired by isolating and fragmenting a broad m/z range which typically contains multiple peptides (38). This m/z range is fragmented repeatedly throughout the experiment, including many times during the chromatographic elution time of a single peptide. Thus, DIA spectra require a different method to identify peptides in a complex multiplexed spectrum (39, 40, 41). As with the previous section, our goal is not to exhaustively review all experimental or computational DIA methodologies; there are numerous variants of DIA which have been developed and optimized for specific needs, and these have been reviewed and benchmarked elsewhere (42, 43). Rather, our focus is to analyze their commonalities and how the basic parameters of DIA might be challenged by the data from SCP.

One common method for DIA data analysis compares observed spectra and chromatographic traces of ions to data from a spectral library (44, 45). Library optimization is a significant computational and experimental topic for the analysis of bulk samples with DIA (46, 47). Thus, it is natural to ask how library optimization can improve the results of SCP-DIA. In single-cell DIA data, we can expect the loss of input ions to create spectra that are noticeably different from bulk DIA data, due to the loss of numerous medium and low intensity fragment ions. This suggests that building spectral libraries from single-cell data would improve peptide identification. Siyal et al. rigorously explored the effect of input sample size for library creation on the effectiveness of DIA peptide identification. They created three libraries of varying sizes: a small library of 1.5 ng (∼10 cells), a medium library of 15 ng (∼100 cells), and a large library of 1 μg (∼5000 cells). A comprehensive spectral library created from multiple bulk experiments with cell lines and human samples was created as a control. They benchmarked the number of proteins identified using a digest of PC-9 cells of various amounts. DIA data was created with 0.75 ng (∼5 cells) and analyzed using the four different libraries. Two thousand three hundred eighty protein groups were identified with the small library, 1908 with the medium library, and 1749 with the large library. Using the comprehensive library, only 107 protein groups were identified. Thus, the protein identifications dramatically increased when using single-cell data to create the library as compared to a library created from ‘small input’. Comparing single-cell data against a typical comprehensive library was completely unsuccessful. Their results were consistent with multiple software tools (Spectronaut and DIA-NN) (48). Other optimizations of DIA for SCP have been mostly experimental (not computational), including adaptations to the chromatography gradient to concentrate ion signals and help them rise above noise (49).

In addition to library-based methods, there are also library-free DIA methods. These tools associate chromatograms for peptides and fragment ions without relying on a library to describe observed fragment ions. Scoring can happen through a pseudospectrum, as with DIA-umpire (40) or directly on chromatograms as with DIA-NN (50). These tools would likely be affected by single-cell data in the same way that label-free quantification (LFQ) chromatogram detection would be due to weaker signal, poor correlation between isotopes, and rough peak elution edges (see Section Label-free quantification).

To help improve DIA accuracy, Demichev et al. (51) incorporated both technical and computational methods in their pipeline that leverage ion-mobility separation. This improved sensitivity by making it easier to associate the chromatograms of precursors with their fragment ions using a multidimensional peak-picking algorithm for ion mobility data. An optimized spectral library and neural network algorithm are used to score precursor-spectrum matches. Their tool was assessed at low sample amounts (10 ng or ∼100 cells). We anticipate that these improvements will be beneficial at the single-cell level.

Possible Solutions to Improve Peptide Identification

Given the recent emergence of SCP, there are no peptide identification algorithms that have been developed specifically for single-cell data. However, with the observations above, there are few obvious optimizations that could be made. 1) All database search algorithms currently rely on the number of annotated peaks that are matched between the observed and theoretical spectrum. The limited number of observed peaks suggests that algorithms should compensate and incorporate expected fragment ion abundances into part of their PSM score. As spectral libraries provide a realistic spectrum that contains intensity differences between peaks, this should become a more significant part of the algorithm. 2) Some database search tools mimic a ‘library search’ by leveraging a MS2 prediction in their score (52). However, this is typically used very late in the algorithm workflow at the stage of p-value assignment. Putting this earlier in the workflow would significantly improve sensitivity. 3) Since there are significant differences between bulk spectra and single-cell spectra, a spectral library should be created from single-cell data, rather than bulk data. If using computational predictions of libraries (52), this should also be trained on the ultra low input of single-cell data to ensure it accurately predicts the substantial loss of fragment ions, as was done in DeepSCP (53). This will improve the accuracy of matches being made between the observed spectrum and the theoretical spectrum. 4) It has been recently shown that PSM identifications significantly improve when using a rescoring tool like Percolator or MokaPot (54, 55). The general advantage of this method is that it uses a rich set of features to calculate a score, instead of the simplistic ‘peak count’ often used in database search tools. The Dart-ID method corroborates this improvement with the inclusion of retention time as a PSM feature (56). Additionally MS2rescore (57), while developed for nontryptic immunopeptidome data, also demonstrates improved scoring with an expanded feature set. Thus, we suggest that peptide identification algorithms for SCP expand their feature space to include a diverse set of metrics and leverage these in semisupervised post hoc scoring. We also note that the differences between single-cell MS2 data and bulk may be instrument-specific and vary based on the physics of fragmentation and ion detection (58).

Peptide Quantitation

After peptides have been identified, they need to be quantified. The method of quantitation is different for label-free and labeled (multiplexed) experimental designs. LFQ methods calculate abundances based on the ion intensity in the MS1 chromatogram (or potentially MS2 in DIA). Multiplexed experiments that use isobaric labeling (TMT) quantitation instead measure the intensity of the reporter ions that appear in MS2 spectra. In this section, we will review work being done for both LFQ and TMT peptide quantitation algorithms.

Label-free Quantification

A peptide’s abundance value is most often calculated by the area of the extracted ion chromatogram (XIC), which reflects the number of ions detected for a peptide. An example XIC for several peptides from single-cell data is shown in Figure 2. Computing the abundance metric varies between software tools. Some tools simply sum intensities, some smooth the data prior to summing (36, 59), and some perform more computationally intense signal processing, e.g., Savitzky-Golay (60). Alternatively, some software tools do not use XICs but rather report the maximal abundance of an ion’s elution (61).

Fig. 2.

Example XIC of peptide ions for single-cell data. Four chromatograms were selected to represent challenging, but not uncommon, scenarios that occur in single-cell data. They demonstrate three challenges with peptide quantification: peak boundary detection, poor correlation within the isotopic peaks, and sporadic and low number of MS1 measurements. These were sampled to represent a range of intensities. Panel A demonstrates a good looking XIC, but the +2 isotope does not correlate well. Panel B shows a much worse chromatogram, where there is coelution interference, and the peak boundaries are difficult to define. Panel D also struggled to define peak boundaries. Panel C and D both have a nonsmooth XIC. XIC, extracted ion chromatogram.

In single-cell proteomics, there are fewer ions for each peptide. Although there are no currently published manuscripts exploring how this affects quantitative accuracy, we recognize several potential problems. Figure 2 demonstrates these problems and shows examples across the intensity range of MS1 peaks. At higher abundances, chromatograms generally look better, but few peptides in SCP have high abundance relative to the norms on which most algorithms are trained and evaluated.

The first challenge is that an ion’s signal is less intense and is closer to the level of noise. This can impact boundary identification and other aspects of peak detection, which require high signal-to-noise. Some algorithms might face a second challenge because of a loss in correlation between the XICs within the isotopic profile. As seen in Figure 2, the most common problem is that correlations between the isotope chromatograms are often poor. If algorithms require strict correlation in the peak detection methods, it is likely that many peptide ions will not be discovered. The next challenge we anticipate with LFQ peptide quantification arises because the ion accumulation time for MS2 spectra is dramatically longer for single-cell data, increasing the spacing between sequential MS1 and yielding fewer measurements across an ion’s elution profile. In SCP, some researchers are shortening the LC gradient in an attempt to tighten a peak’s elution and increase signal. However, this exacerbates the problems associated with having too few measurements across an elution profile (62, 63). A final subtle challenge for single-cell peptide quantification is that the time interval between successive MS1 scans can be highly irregular. This comes from both a variable number of MS2 spectra acquired as well as the widely varying time for ion accumulation. To our knowledge, only the Skyline algorithm accounts for this type of temporal irregularity in calculating ion quantity, by resampling the chromatogram (60). Other algorithms which simply sum MS1 scans or do a kernel smoothing would be significantly affected in quantitative accuracy.

Match Between Runs

Match between runs (MBR) was developed to address situations where ions were measured in MS1 spectra but not selected for MS2 identification (36, 64). To rescue these potential peptides, MBR transfers the peptide identification from another run based on the ion’s mass and aligned retention time, thus improving peptide identification coverage within an experiment. This technique is part of many software tools as an intermediate step between peptide identification and peptide quantification. One of the most important advances to be made to MBR was the development of a statistical confidence measure of each transfer, facilitating FDR-based cutoffs and improving accuracy (65). As MS2 identifications are generally more stochastic with single-cell data, MBR plays a relatively larger role in peptide identification/quantification.

Two recent papers have adapted the MBR technique for the unique challenges of SCP. Woo et al. (66) recently developed TIFF to improve sensitivity and accuracy. The TIFF experimental workflow uses FAIMS to filter out background and +1 ions to increase signal for peptide ions. Moreover, the TIFF algorithm incorporates the FAIMS CV into the MBR match to supplement the traditional m/z and retention time. By including a third metric into the matching criteria, they report a significant increase in sensitivity and accuracy. A second tool, IceR, seeks to improve MBR for SCP by improving ion detection in ultra-low input samples (67). As MBR was developed for bulk data with rich ion features, most peak detection methods require molecular ion peaks in MS1 spectra to have peaks that correspond to the isotopic envelope. However, ion abundances in single-cell data are so low that isotopic peaks sometimes are not detected or have poor correlation (see Fig. 2 and discussion above). Thus, the IceR implementation of MBR does not require an isotopic envelope for peak detection, which significantly reduces missing data for peptide quantification.

Tandem Mass Tag

In multiplexed isobaric labeling with tandem mass tags (TMTs), each sample is labeled with an isobaric tag. After labeling each individual sample, all samples are combined and analyzed via LC-MS/MS. Due to the unique placement of the isotopes, each tag produces a unique fragment ion in MS2 spectra which is used to measure abundance (i.e., reporter ion) (68). Multiplexed experiments are especially valuable in single-cell proteomics, as combining samples increase signal intensity and yield a better MS2 spectrum. To further increase signal, the SCoPE-MS method utilizes a carrier proteome containing dozens to hundreds of cells to boost the signal (69). However, caution must be used when implementing the carrier proteome. While the increase in ions improves peptide identification, there is a tradeoff with protein quantitation accuracy (70, 71). Cheung et al. (70) recommend a carrier limit of ∼20 to 50× along with SNR filtering, but note that higher carrier might be used if instrument setting and postanalysis filtering is carefully optimized. They found that high carrier amounts significantly under samples the number of ions for other reporter channels, making it more difficult to distinguish true differences between single-cells and limiting quantitative accuracy. Similarly, Ye et al. (71) suggest the most accurate quantitation is achieved with a carrier limit of 42×, but 98× could achieve an acceptable balance between identification and quantification. Thus, both manuscripts recommend a carrier proteome limit of ∼50×, while allowing up to ∼100×.

Beyond undersampling of the reporter ion channels, a high carrier proteome also distorts the quantitation of nearby channels due to impurities in the reagent. To demonstrate the impact of this bleedover of TMT signal from carrier channel to other channels, we analyzed a recently published unique TMT experiment (72). While this experiment was not designed specifically to study the effects of the carrier proteome, it omits a large number of reporter ion channels and thus provides important insight into the magnitude of bleedover. Figure 3 shows the intensity of all reporter ion channels across 6451 high confidence PSMs. Channel 126 is the boost of 10 ng (∼100 cells), and 127N is a reference of 0.5 ng (∼5 cells). Nine channels were used for 0.1 ng (single-cell amounts) of three different cell types: channels 130N, 131C, and 133N are epithelial cells, channels 130C, 132N, and 133C are immune cells, and channels 131N, 132C, and 134N are endothelial cells. Importantly, channels 127C, 128N, 128C, 129N, and 129C were not used and thus any signal detected in them is noise. When using a large number of cells in the 126 channel, researchers typically leave 127C blank, as a ¹³C impurity in the 126 reagent would show up as signal in 127C. As shown in Figure 3, channel 127C has a high signal, similar to the abundance of the reference channel. Because this dataset left four other channels blank, we are able to see whether the bleedover from the carrier channel and reference channel extend beyond the expected 127C. Channel 128C has high intensity that is most likely to be attributed to a double ¹³C impurity in the 126 carrier channel. The intensity in 128N is likely to be a ¹³C impurity from the 127N reference channel. Indeed, both 128N and 128C have intensity similar to 130N, which is data for a single epithelial cell. This bleedover beyond the one standard blank, 127C, was also noted by Ye et al. (71). Thus, with a 100× carrier channel, we see significant bleedover into many channels that would typically be used for experiments.

Fig. 3.

Measured signal intensity in blank TMT channels. The intensity of each TMT reporter ion is plotted for 6451 PSMs filtered to 1% FDR. Shown in rose colored boxes are two channels which are not a single cell: channel 126 is used for the 100× carrier, and channel 127N is used for the 5× reference sample. Shown in gray boxes are the blank channels. Channels 127C, 128N, 128C, 129N, and 129C were omitted from the experimental design, yet they contain noticeable measured signal. Shown in yellow, blue, and red boxes are channels for the single cells from three different cell lines, as described in reference (72). FDR, false-discovery rate; PSM, peptide spectrum match.

Several methods are currently used to improve quantitative accuracy in TMT-labeled SCP, however, we note that they are experimental and not computational improvements. First, in an attempt to remove the effect of coisolated precursor ions, researchers have utilized the SPS-MS3 method developed in bulk studies (73, 74). Here multiple intense fragment ions in MS2 are chosen for MS3. This second round of isolation and fragmentation removes the contaminating signal from potential cofragmented precursor ions, avoiding the quantitative inaccuracies caused by ‘ratio compression’ (75). Although this has shown to be very impactful for quantitative accuracy in bulk studies, there are several challenges with regards to SCP. First, creating an MS2 and MS3 spectrum for each peptide ion results in significantly reduced peptide/protein coverage (74). In an attempt to overcome this time cost, researchers have explored using the faster linear ion trap (76). Although this improves peptide identification, there are two drawbacks: 1) the inherently lower mass accuracy of the ion trap limits the number of TMT channels that can be used in an experiment and 2) the protein quantification in the ion trap had a substantially higher %CV than an Orbitrap. A second challenge for using MS3 with SCP is that the extremely limited number of ions present in single cell data means that secondary selection for MS3 spectra will potentially not have enough ions for meaningful quantitation, especially for experiments using a large carrier proteome. Third, as shown in Figure 3, quantitative accuracy of SCP is significantly impacted by the bleedover of the carrier channel, which is not resolved by SPS-MS3. Therefore, a second essential experimental design consideration for multiplexed SCP is the inclusion of a blank channel within the multiplexed design. The intensity in this blank channel is a reliable measure of noise, which may arise for myriad cases. Using the blank channel to quantify noise allows for a simple calculation of whether specific PSMs have sufficient signal/noise to be considered quantifiable.

Possible Solutions to Improve Peptide Quantification

While there have been various implementations of MBR designed to account for a specific difference in single-cell data (66, 67), we lack an in-depth analysis of how MBR is performing on single-cell data. One fundamental difference that has not been accounted for yet is that MBR was developed for bulk data, where ions are plentiful and time is limited. This contrast leads to data acquisition where it is not possible to select every peptide for MS2 fragmentation. Single-cell data is the opposite. The limiting factor is the paucity of ions. MBR could be greatly improved for single-cell data by accounting for base assumptions that were made for bulk data that are not true for single-cell data.

In proteomics, there is a long history of separately defining the limit of detection from the limit of quantification, especially in targeted proteomics (77, 78). Some analytes are detectable, but not reliably quantifiable. We believe that this distinction will be very important for SCP, as the ultra low sample input often means that analytes are between the limit of detection and limit of quantification. Thus, researchers may need to be more obvious and direct in the reporting of SCP results, bluntly stating that many peptides are detectable but not quantifiable. This may be unpopular as it would reduce the overall depth of proteome coverage reported in a manuscript, but we believe it is an essential part of research integrity. Towards this end, the SCPCompanion tool is designed for TMT multiplexed single-cell data and calculates a signal/noise ratios for all PSMs based on the instrument’s native S/N calculation (70). SCPCompanion suggests a cutoff used on filtering data which is nonquantifiable. Additionally, we strongly suggest that ‘blanks’ be more heavily relied upon for experimental design; questionably quantified analytes are easy to find for both TMT and LFQ studies that include blanks to explicitly measure noise. Although blanks are often mentioned in the experimental design section, many manuscripts are curiously silent about how these are utilized in data analysis.

Protein Identification and Quantitation

In bottom-up proteomics, the measured analyte is a peptide. Yet, the ultimate goal of proteomics is often to identify and quantify proteins. In proteolytic digestion, the direct connection between peptides and protein is lost. Inferring which peptides belong to which proteins and determining a protein’s quantitative value has both theoretical and algorithmic challenges (79, 80). We will give a brief overview of how protein identification and quantitation work—pointing the reader to more thorough reviews and benchmarks (77, 81, 82, 83)—and then discuss the challenges for single-cell proteomics.

Protein identification algorithms group peptide sequences into a protein. If each peptide uniquely mapped to a single protein, this would be a trivial step. Since peptides often belong to multiple proteins, assigning peptides to proteins becomes a much more complex problem (84, 85). The most common method used for protein identification is parsimony, which derives the minimal protein list that is sufficient to account for all identified peptides (86). Two other solutions are to completely filter out peptides that map to multiple proteins or to assign shared peptides to the protein group with the greatest number of matching peptides (the razor solution). However, condensing peptide measurements into a single protein report may potentially obscure biologically relevant differences between related protein isoforms (87, 88). To avoid these problems with protein-based quantification, some researchers have advocated for working more directly with peptide abundances to identify significant differences between samples (89, 90, 91, 92). As peptide-centric methods have yet to be used in SCP, this review will focus on protein-level quantification.

Proteins are quantified by summarizing peptide level abundances into a single protein value. The most common types of methods are sum_all, topN, and median value. As suggested by their names, these methods combine peptide abundances into a single, protein-level abundance using basic mathematical operations. For example, the ‘sum_all’ method calculates the protein abundance by summing all peptide abundances; the ‘median value’ method calculates the protein abundance from the median abundance value of constituent peptides. There are many differences in specific implementations of each of these general types, e.g., simple median versus Tukey’s median polish; sum all versus iBAQ. The quantitative accuracy of various methods and implementations for bulk proteomics has been reviewed elsewhere (93). Currently, there is no consistently used method within the SCP community (11).

Challenges

In SCP, the sparseness of the peptide quantitative data matrix is likely to make protein quantification a significant challenge. Regardless of the details of algorithmic implementation, these methods implicitly require a large number of consistently quantified peptides per protein for stable and reliable summarization (87). Although algorithmic details of protein summarization are poorly described for many software tools, we review below how the common methods for sum_all, topN, and median might be challenged with SCP data. The problems outlined below are not unique to SCP. But they are much more prominent and should be directly addressed as they are no longer simply an ‘edge case.’

One of the most significant adjustments that algorithms will need to make is the expectation on how many peptides can contribute to the summarization formula and the level of sparsity in peptide quantification tables. For example, the topN method is most often implemented as Top3. However, this begs the question for how often proteins in SCP have at least three peptides, and is there consistent data for these three peptides? If choosing a Top3 method and not doing an explicit peptide abundance imputation, there is an implicit imputation of zero for any missing data.

To demonstrate challenges with quantitation, we use six LC-MS/MS technical replicates of a HeLa digest (14). We quantified the number of consistent peptides per protein, where consistent peptides are defined as being present in 6/6 replicate runs. We chose this strict definition of ‘consistent’ as it is most clear for the discussion, although we recognize that algorithms may choose a different cutoff or have no cutoff at all. We found that of the 1420 proteins, 29% did not have any consistent peptides, 24% had one consistent peptide, 11% had two consistent peptides, 8% had three consistent peptides, and 28% had more than three consistent peptides (Fig. 4).

Fig. 4.

The number of consistent and inconsistent peptides in each protein. Proteins were categorized based on the number of consistent peptides (x-axis) and subdivided into the number of inconsistent peptides (colors within the stacked bar). For example, the first stacked bar counts how many proteins have no consistent peptides (∼400 proteins) and then subdivides it into the number of inconsistent peptides each protein contains. The majority have one inconsistent peptide (∼250 proteins in orange), meaning that proteins that do not contain any consistent peptides usually; only have one inconsistent peptide. Accurately quantifying such proteins would be very difficult.

We discuss the potential for volatility in protein quantification with three examples where there are fewer than three consistent peptides. In each example, the choice of summarization method (sum_all, topN, or median) has a dramatic impact on the reported protein quantitation value. Since there are not three consistent peptides in these examples, algorithms are forced to deal with inconsistent peptides which have missing data. We note that very few software tools describe how peptide (in)consistency is addressed, i.e., how are peptides filtered prior to summarization. They are similarly unclear about whether imputation is used on the peptide quantification matrix to partially ameliorate missing data prior to protein summarization. As we show in each of the demonstrative examples below, these decisions are critical. In examining the examples, assume that these are the final peptide quantitation values used as input to the protein summarization method. Any imputation, normalization, or filtering has already happened.

Figure 5A shows the metabolic protein Aconitase (ACO2), which is supported by two inconsistent peptides. Each peptide has similar amounts of missingness, but the missing values occur in different samples. In only one run (run5), both peptides are observed. Consider how sum_all and a median summarization method would perform differently on this protein. Sum_all would add the quantitative value of both peptides and report that the protein is most abundant in run5. We note again for emphasis that the sum_all method here does an implicit imputation of zero for the intensity of peptides with missing data. The median method would report nearly the opposite. With two values in run5, the median method would average the two peptide abundances, and therefore, rank run5 near the bottom compared to other runs in the experiment.

Fig. 5.

Proteins that contain inconsistent peptides. Each protein illustrates how quantitation methods are not designed to handle sparse data. The protein in panel A only contains inconsistent peptides and will be quantified differently depending on the summarization method used. The proteins in B and C illustrate that filtering out inconsistent peptides will drastically alter the results of quantitation.

Figure 5B shows ribosome related protein RRS1 which is supported by one consistent peptide and two inconsistent peptides. A topN or sum_all method would nominally consider all three peptides in creating a protein abundance. If all three peptides are used for quantitation, then the protein is highly abundant in run1, the second most abundant in the whole experiment. However, peptide VEEAIVAR is only present in 2/6 runs. If this peptide were to be filtered out prior to summarization due to inconsistent quantification, then run1 would rank near the bottom of the experiment. If this peptide were dropped, and then a median methodology was used, run1 would be at the very bottom. As mentioned above, most software tools do not describe this step of protein quantification in manuscripts or supplemental methods. Thus, it is not trivial to determine how scenarios such as this are resolved.

Figure 5C shows the peptide quantitation table for the metabolic protein phosphoglucomutase (PGM1). This protein has six identified peptides; two peptides are consistent and observed in every run within the experiment. Four peptides are inconsistent across the experiment. Consider how top3 would perform in this example. Several inconsistent peptides have the highest abundance, which would suggest their inclusion in the top3 calculation. Depending on how the three peptides are chosen (or perhaps fewer if three meritorious peptides are not found), the protein could be most abundant in run 1, 5, or 6. Similar volatility exists for this example protein using the sum_all or median method.

The thought experiment described above is provocative. Indeed, as standard proteomics software tools lack any clear documentation on how these situations are resolved, it is difficult to know how protein quantification is affected in reality by incomplete and sparse peptide quantification matrices. Fortunately, the scp library in R has implemented this step of protein rollup using a variety of methods (15). Using scp and the peptide matrices from Figure 5, we created the abundances for ACO2, RRS1, and PGM1 (Fig. 6) using the sum, top3, and median methods. As suggested by the thought experiment, we observe significant disagreement as to which run contains the most abundant protein. For example, in the quantification of PGM1, run6 could be the highest or lowest in abundance depending on the method that was used.

Fig. 6.

Various quantitation methods performed on proteins containing inconsistent peptides. The 'peptides' column shows the abundance of individual peptides for each example protein (ACO2, RRS1, and PGM1). The columns 'medians', 'sums', and 'top3′ show the quantitation result for each respective method, as calculated by the scp package. Note the difference between each method, particularly the median method and sum/top3.

One final note on protein quantification is the impact of imputation, which can be applied to either the peptide and/or protein abundance matrices. Computational methods for imputation are diverse and have been compared and reviewed elsewhere (94, 95, 96, 97, 98, 99), and the choice of method can affect the downstream interpretation of the experiment (100). Historically, many benchmarking comparisons and tool development efforts are pursued at the level of protein imputation (101, 102, 103). However, as seen in Figures 4 and 5, missing data is prevalent in peptide quantitation and has a significant impact on the resulting protein abundance calculation. Since missing data is more common in SCP than bulk proteomics, strategies should be evaluated in SCP for both peptide and protein-level imputation.

Possible Solutions

Protein quantitation methods for single-cell data must consider whether or not peptides making up a protein are consistent, since the more consistent a peptide is, the more reliable the quantitation will be. The proteins shown in Figures 5 and 6 demonstrate a volatility, depending on the type of protein summarization method employed. The confounding nature of these example proteins is quite common in SCP, with the vast majority of protein identifications having fewer than three consistent peptides. The importance of considering peptide consistency was demonstrated in a recent paper by the Vitek group, in which they developed an automated statistical approach that performs protein quantification using an informative subset of features (104). Feature is a generic term that refers to precursor ions in the case of DDA data, transitions in SRM data, and fragments in DIA data. Informative features are features that have few missing values, and for which intensity values are consistent with most of the protein intensity profiles across runs. Their method detects features that have inconsistent patterns, which can then be investigated and removed, facilitating more accurate peptide-level data. The suggested data presence criteria that comes from this method depends on the experimental design. Given that peptide quantitation matrices in SCP are much more sparse than in traditional bulk proteomics, we strongly suggest research into how peptide consistency affects protein quantitation. We also advocate for more transparency and better description of the summarization method in manuscripts and documentation of various software tools.

Conclusion

Single-cell proteomics has been making great strides in sample preparation, separation technologies, and measurement speed and sensitivity. However, the research to improve algorithms that interpret the resulting data is just beginning. This review considers the main types of algorithms used for peptide and protein identification and quantification, challenges of applying them to single-cell data, and possible solutions. Importantly, we note that every part of this identification/quantification workflow faces significant challenges, and thus all are targets for improvement. We note that none of the problems described in this review are exclusive to SCP. Rather, they were often ‘edge cases’ or rare occurrences that were not explicitly treated in bulk data. However, for SCP, the above challenges are major computational roadblocks that need to be addressed.

Peptide identification algorithms rely heavily on the number of peaks matched and the intensity of peptide ions being far above background noise. While these conditionals are met in bulk data, single-cell data has fewer ion peaks and the S/N ratio is substantially compressed. Some possible solutions are to 1) use a spectral library created from single-cell data, 2) use predicted MS2 spectra trained on single-cell data, 3) use semisupervised post hoc scoring tools, and 4) expand the feature space to include a diverse set of metrics. Peptide quantitation is also challenged as a result of fewer ions being detected, and the signal of peptide ions is much closer to the level of noise in comparison to bulk data. LFQ could struggle with multiple facets of feature finding and accurate quantitative calculation. TMT methods incorporate a carrier channel to improve peptide identification, however this impairs quantitative accuracy. The data generation methods have been exploring possible solutions, yet these have not often investigated the potential tradeoffs between identification and quantification, with several choices improving identification yet harming quantification. For example, shortening LC gradients improves ion signals but limits MS1 sampling which complicates quantitation. Moving forward, we need a greater integration between data generating and data analysis teams to explore viable solutions. Finally, protein quantitation algorithms rely on the presence of several consistent peptides. However, in single-cell data, we see a considerable amount of proteins with inconsistent peptides that the peptide quantitation algorithms are not robust enough to deal with them. There needs to be research into protein quantitation algorithms that can handle sparse data.

Data Availability

The data used in these analyses are six replicates of 0.2 ng. The data is uploaded to Massive (MSV000087524), and all code used to generate figures and analyze results can be found in the github repository, https://github.com/PayneLab/SCP_comp_review.

Conflict of interest

The authors declare no competing interests.