Assigning mutational signatures to individual samples and individual somatic mutations with SigProfilerAssignment

Abstract

Analysis of mutational signatures is a powerful approach for understanding the mutagenic processes that have shaped the evolution of a cancer genome. Here we present SigProfilerAssignment, a desktop and an online computational framework for assigning all types of mutational signatures to individual samples. SigProfilerAssignment is the first tool that allows both analysis of copy-number signatures and probabilistic assignment of signatures to individual somatic mutations. As its computational engine, the tool uses a custom implementation of the forward stagewise algorithm for sparse regression and nonnegative least squares for numerical optimization. Analysis of 2,700 synthetic cancer genomes with and without noise demonstrates that SigProfilerAssignment outperforms four commonly used approaches for assigning mutational signatures. SigProfilerAssignment is freely available at https://github.com/AlexandrovLab/SigProfilerAssignment with a web implementation at https://cancer.sanger.ac.uk/signatures/assignment/.

Somatic mutations accumulate in the genomes of all cells of the human body throughout an individual’s lifetime^1,2. These mutations arise from different endogenous and exogenous mutational processes, with each process generating a characteristic pattern of mutations, known as a mutational signature³. By leveraging the vast amounts of high-throughput DNA sequencing data generated over the last two decades, distinct mutational signatures have been elucidated from various cancer types^4,5 and normal somatic tissues^6–9. Sets of mutation-type specific reference signatures have been developed and deposited in the Catalogue of Somatic Mutations in Cancer (COSMIC) database^4,10, including signatures of single base substitutions (SBSs), doublet base substitutions (DBSs), small insertions and deletions (IDs), and copy number alterations (CNs). In practice, mutational signatures have been widely utilized in research and clinical settings, including for detecting previously unappreciated cancer predispositions^11,12, pathogenic classification of germline variants¹³, clinical management of cancer patients¹⁴, and identifying sensitivity to anti-cancer therapeutics¹⁵.

There are at least two distinct approaches for analyzing mutational signatures. De novo extraction is an unsupervised machine learning approach that allows identifying the patterns of known and previously unknown mutational signatures¹⁶. This type of analysis is predominately used for deriving reference signatures as it requires large cohorts, generally more than 100 samples and, usually, many thousands of samples. In contrast, refitting of mutational signatures is a numerical optimization approach that allows the assignment of known (in most cases, reference) signatures to an individual sample by quantifying the number of mutations attributed to each signature operative in that sample. While refitting cannot identify or quantify the activities of previously unknown mutational signatures, this approach is widely applied to small cohorts and for clinical samples where evaluations are almost exclusively performed for an individual cancer patient¹⁷.

In the past decade, multiple tools for refitting known mutational signatures were developed, including deconstructSigs¹⁷, MutationalPatterns^18,19, sigLASSO²⁰, and SignatureToolsLib^5,21. The majority of these tools lack online interface and provide support almost exclusively for substitution signatures. Further, most tools for refitting of known mutational signatures have never been compared and no existing tool supports probabilistically assigning mutational signatures to somatic mutations. To address these limitations, here, we present SigProfilerAssignment, a comprehensive computational tool for assigning mutational signatures to individual samples and individual somatic mutations (Fig. 1). In contrast to other tools, SigProfilerAssignment provides desktop and online support for all types of mutational signatures, including the COSMIC sets of reference SBS, DBS, ID, and CN signatures. In addition to COSMIC reference signatures, SigProfilerAssignment supports assignment of de novo extracted mutational signatures as well as of a user provided set of custom mutational signatures. Our benchmarking based on 2,700 simulated cancer genomes demonstrates that SigProfilerAssignment outperforms other commonly used tools on simulation data with and without noise.

Figure 1. Assigning known mutational signatures to an individual sample and individual mutations with SigProfilerAssignment.

SigProfilerAssignment supports input data in a standard format (VCF, MAF, or text) and it allows assigning a set of known signatures (e.g., ones from the COSMIC database) to an a) individual sample and b) probabilistically to an individual somatic mutation. Note that the probabilistic assignment of mutational signatures to an individual somatic mutation is only possible if a user provides a list of individual mutations (e.g., VCF file) for the examined sample instead of a mutational vector, as a mutational vector lacks information for individual mutations.

Given a set of known mutational signatures and a set of mutations in a cancer genome, both classified under the same mutational schema^3,22, SigProfilerAssignment identifies the number of mutations caused by each signature in that cancer genome (Fig. 1a). Mathematically, a mutational schema can be represented as a finite alphabet $\mathrm{\Xi }$ of mutation types containing a total of $\xi$ letters. Here, a mutational signature is defined as a probability mass function with domain the alphabet $\mathrm{\Xi }$.

In vector notation, a mutational signature can be denoted as $\overrightarrow{\mathit{s}}={\left[s_1,s_2,\dots ,s_{\xi }\right]}^{\mathrm{T}}$, where $s_k,1\le k\le \xi$, is the probability for the mutational signature, $\overrightarrow{\mathit{s}}$, to cause mutations of type corresponding to the k^th letter of the alphabet $\mathrm{\Xi }$. Since a mutational signature is a probability mass function, $0\le s_k\le 1$ and ${\sum }_{k=1}^{\xi }{s}_k=1$. As such, a set of known $n$ mutational signatures can be expressed as a signature matrix, $\mathit{S}\in {\mathbb{R}}_{+}^{\xi \times n}$, where $\mathit{S}=[{\overrightarrow{\mathit{s}}}^1,{\overrightarrow{\mathit{s}}}^2,\dots ,{\overrightarrow{\mathit{s}}}^n]$. Further, a set of mutations in a cancer genome can be defined as $\mathit{v}:\mathrm{\Xi }\to {\mathbb{N}}_{+}^{\xi }$. In vector notation, a set of mutations in a cancer genome $\overrightarrow{\mathit{v}}={\left[v_1,v_2,\dots ,v_{\xi }\right]}^{\mathrm{T}}$, where $v_k,1\le k\le \xi$, reflects the number of mutations in that cancer genome of the mutation type corresponding to the k^th letter of the alphabet $\mathrm{\Xi }$. SigProfilerAssignment takes as an input a signature matrix, $\mathit{S}$, and a set of mutations, $\overrightarrow{\mathit{v}}$, to output a column vector of activities $\overrightarrow{\mathit{a}}={\left[a_1,a_2,\dots ,a_n\right]}^{\mathrm{T}}$, where $a_t\in {\mathbb{N}}_0^n,1\le t\le n$, corresponding to the number of somatic mutations attributed to the t^th mutational signature. The underlying assumption of assigning mutational signatures is that the mutations within a sample can be approximated as a superposition of known mutational signatures and their activities:

$$\overrightarrow{\mathit{v}}\approx \mathit{S}\overrightarrow{\mathit{a}}$$ (1)

Thus, subject to $\overrightarrow{\mathit{a}}\ge 0$, one needs to derive the vector $\overrightarrow{\mathit{a}}$ that best fits the provided input data. To solve this optimization problem, SigProfilerAssignment uses a custom implementation of the forward stagewise algorithm²³ and it applies nonnegative least squares (NNLS)²⁴, based on the Lawson-Hanson method²⁴:

$$\underset{\overrightarrow{\mathit{a}}\ge 0}{\mathrm{m}\mathrm{i}\mathrm{n}}\parallel \overrightarrow{\mathit{v}}-\mathit{S}\overrightarrow{\mathit{a}}{\parallel }_2^2$$ (2)

The algorithm starts by first computing a minimum relative error, ${\mathrm{ϵ}}_{\mathrm{m}\mathrm{i}\mathrm{n}}=\frac{\parallel \overrightarrow{v}-\mathit{S}\overrightarrow{\mathit{a}}{\parallel }_2^2}{\parallel \overrightarrow{v}{\parallel }_2^2}$, by deriving the nal nonnegative vector $\overrightarrow{\mathit{a}}$ for the complete set of all reference signatures, $\mathit{S}$, using equation (2). This minimum error provides the best possible explanation of the data, but it also results in overfitting as all available signatures are utilized. Next, the tool uses steps for removing and adding signatures based on the backward and forward stepwise algorithms, respectively²³. First, signatures are removed by employing a backward stepwise algorithm²³ (Algorithm 1). Specifically, each signature from the reference signature set, $\mathit{S}$, is removed iteratively and the remaining signature set, $\widehat{\mathit{S}}$, is attributed to the sample $\overrightarrow{\mathit{v}}$ by applying equation (2). The increase in the relative error, ${\mathrm{ϵ}}_{\mathrm{j}}=\frac{\parallel \overrightarrow{v}-\widehat{\mathit{s}}\overrightarrow{\mathit{a}}{\parallel }_2^2}{\parallel \overrightarrow{v}{\parallel }_2^2}-{\mathrm{ϵ}}_{\mathrm{m}\mathrm{i}\mathrm{n}}$, due to removing a signature is calculated by removing the j^th signature from $\mathit{S}$. The signature with the least relative increase in error rate is removed from the signature set, $\mathit{S}$, provided that the increase is less than a specific threshold (default value of 0.01). After the final removal of the signature with least relative error rate increase, the minimum relative error, ${\mathrm{ϵ}}_{\mathrm{m}\mathrm{i}\mathrm{n}}$, and the set of signatures, $\mathit{S}$, are updated to reflect this removal. The removal step is repeated until all signatures satisfying the conditions are removed from $\mathit{S}$. The removal steps are followed by addition steps based on the forward stepwise algorithm²³ (Algorithm 1). Specifically, each of the previously removed reference signatures is added back iteratively to $\mathit{S}$ and the new signature set, $\widehat{\mathit{S}}$, is fit for the sample $\overrightarrow{\mathit{v}}$ by applying equation (2). Thus, the decrease in the relative error, ${\mathrm{ϵ}}_l={\mathrm{ϵ}}_{\mathrm{m}\mathrm{i}\mathrm{n}}-\frac{\parallel \overrightarrow{\mathit{v}}-\widehat{\mathit{S}}\overrightarrow{\mathit{a}}{\parallel }_2^2}{\parallel \mid \overrightarrow{v}{\parallel }_2^2}$, due to adding a signature is calculated by adding the l^th signature to $\mathit{S}$. The signature with maximum relative decrease of the error rate is added back to the signature set, $\mathit{S}$, provided that the increase is more than a specific threshold (default value of 0.05). After the final addition of the signature with most relative rate decrease, the minimum relative error, ${\mathrm{ϵ}}_{\mathrm{m}\mathrm{i}\mathrm{n}}$, and the set of signatures, $\mathit{S}$, are updated to reflect this addition. The addition step is repeated until all signatures satisfying the conditions are added back to $\mathit{S}$. Lastly, the addition and removal steps are repeated until convergence, where no signature is added or removed from the list of signatures (Algorithm 1).

In addition to quantifying the activity of each mutational signature, SigProfilerAssignment also assigns known signatures to individual mutations (Fig. 1b) based on their specific mutational context:

$$p_k^t=\frac{s_k^t{a}_t}{[\mathit{S}\overrightarrow{\mathit{a}}{]}_k}$$ (3)

where, $p_k^t$ represents the probability of a mutation corresponding to the 𝑘^th letter of the alphabet $\mathrm{\Xi }$ being caused by the 𝑡^th signature in the sample; $s_k^t$ is the probability of the 𝑡^th signature to cause mutation corresponding to the 𝑘^th letter of the alphabet $\mathrm{\Xi };a_t$ is the number of mutations attributed to the 𝑡^th mutational signature; and $[\mathit{S}\overrightarrow{\mathit{a}}{]}_k$ is the value of the 𝑘^th element of the vector obtained by the matrix multiplication of the signature matrix, $\mathit{S}$, and the derived signature activities, $\overrightarrow{\mathit{a}}$.

To evaluate the performance of SigProfilerAssignment and another four commonly used tools for refitting mutational signatures^5,17–21, we performed a comparative benchmarking using a previously generated independent synthetic dataset¹⁶ (Fig. 2). The dataset encompasses the SBS patterns of 2,700 simulated cancer genomes, corresponding to 300 tumors from nine different cancer types, generated using 21 different COSMIC reference signatures. To emulate a typical refitting of mutational signatures, the complete set of 79 COSMICv3.3 SBS signatures was used as input. The mutational signature activities obtained by each tool were compared against the ground truth activities used to synthetically generate these samples. Three different levels of random noise (0%, 5%, and 10%) were tested to assess the stability of the different algorithms in a real biological context. To evaluate the accuracy of the signature refitting, we calculated sensitivity, specificity, and F₁ score (Methods). In addition, we also examined the runtime and memory utilization of each tool.

Figure 2. Benchmarking of SigProfilerAssignment and four other tools for assigning mutational signatures.

Each tool was evaluated using 2,700 synthetic cancer genomes generated using 21 different COSMIC reference mutational signatures. All COSMICv3.3 signatures were used as the input set of known mutational signatures. a) Three different levels of non-systematic random noise (0%, 5%, and 10%) were used to evaluate the precision (x-axes), sensitivity (y-axes), and F₁ scores (harmonic mean of precision and sensitivity; red dotted lines) of each tool. b) Computational benchmarking based on CPU elapsed time (x-axis; log-scaled) and maximum memory usage (y-axis) for each tool.

Our synthetic benchmarking revealed that SigProfilerAssignment outperforms all other approaches for the examined noise levels (Fig. 2a). For 10% random noise, only SigProfilerAssignment obtained an F₁ score >0.90. In all cases, SigProfilerAssignment exhibited a high precision while showing an improved sensitivity compared to other approaches (Fig. 2a). In terms of computational performance, SigProfilerAssignment processed the 2,700 samples within 9.6 minutes (0.21 seconds per sample). Only the standard mode of MutationalPatterns generated results substantially faster (Fig. 2b). However, MutationalPatterns’ standard mode exhibited sub-optimal performance, with a significant drop in precision for all noise levels, likely due to overfitting of the input data (Fig 2a)¹⁹. This issue has been addressed in the most recent version of MutationalPatterns with the addition of a strict mode¹⁸, albeit with a significant computational performance cost (Fig. 2b). Other approaches limit overfitting by implementing different penalties based on the L1 error (viz., sigLASSO)²⁰ or the sum-squared error (viz., deconstructSigs)¹⁷, as well as post-hoc filters based on the percentage of the total number of mutations attributed to a given signature (viz., deconstructSigs and SignatureToolsLib)^5,17. No significant memory requirements were observed for any of the tools (Fig. 2b).

Assigning mutational signatures to individual samples provides an opportunity to identify the processes responsible for somatic mutations on a sample-by-sample basis. Considering our synthetic benchmarking, SigProfilerAssignment stands out as the most precise and sensitive tool while maintaining high computational performance and bringing novel capabilities. To the best of our knowledge, SigProfilerAssignment represents the first computational tool for assigning signature probabilities to individual mutations, which can allow uncovering the mutational processes responsible for specific driver genomic alterations leading to tumor evolution. SigProfilerAssignment is also the first tool that supports assignment of the recently developed copy number signatures²⁵, which are good predictors of clinical survival^25,26.

In summary, SigProfilerAssignment provides a novel computational package and an accessible online interface to accurately assign known mutational signatures to an individual cancer and individual somatic mutations, thus, enabling users to ascertain the mutational processes operative in a cancer genome.

Algorithm 1:

Assigning mutational signatures to samples with SigProfilerAssignment

	Input: $\overrightarrow{\mathit{v}}\in {\mathbb{N}}_{+}^{\xi \times 1}$ (a vector corresponding to a set of mutations in a sample) and $\mathit{S}\in {\mathbb{R}}_{+}^{\xi \times 1}$ (a matrix corresponding to a set of $n$ known mutational signatures)
	Output: $\overrightarrow{\mathit{a}}\in {\mathbb{N}}_{+}^{n\times 1}$ (the vector reflecting the activities of the $n$ known signatures in sample $\overrightarrow{\mathit{v}}$)
1:	${ϵ}_{\mathrm{m}\mathrm{i}\mathrm{n}}$, $\overrightarrow{a}=\text{calcNNLS}(\overrightarrow{v},\mathit{S})$
	${\mathit{S}}^{\text{all}}=\mathit{S}$
2:	While FLAG = True:
3:	${ϵ}_{\mathrm{m}\mathrm{i}\mathrm{n}}$, $\mathit{S}=\mathit{r}\mathit{emoveSignatures}(\overrightarrow{v},\mathit{S},{ϵ}_{\mathrm{m}\mathrm{i}\mathrm{n}})$
4:	${ϵ}_{\mathrm{m}\mathrm{i}\mathrm{n}}$, $\mathit{S}=\mathit{addSignatures}(\overrightarrow{\mathit{v}},{\mathit{S}}^{\text{all}},\mathit{S},{ϵ}_{\mathrm{m}\mathrm{i}\mathrm{n}})$
5:	Set FLAG = False if $\mathit{S}$ remains constant and there is no addition or removal of signatures
	END While
6:	${ϵ}_{\mathrm{m}\mathrm{i}\mathrm{n}}$, $\overrightarrow{\mathit{a}}=\mathit{c}\mathit{a}\mathit{l}\mathit{c}\mathit{N}\mathit{N}\mathit{L}\mathit{S}(\overrightarrow{v},\mathit{S})$
7:	Return $\overrightarrow{\mathit{a}}$
8:	FUNCTION removeSignatures $(\overrightarrow{\mathit{v}},\mathit{S},{ϵ}_{\mathrm{m}\mathrm{i}\mathrm{n}})$
9:	While FLAG = True:
10:	For $\mathit{j}$ in 1 to size($\mathit{S}$, 2) do	// loop from 1 to the total number of signatures in $\mathit{S}$
11:	$\stackrel{\mathit{ˆ}}{\mathit{S}}=\mathit{S}[:,-\mathit{j}]$	// remove the jth signature from $\mathit{S}$
12:	$\in \left[j\right]$, ${\overrightarrow{\mathit{a}}}_{\mathit{j}}=\mathit{c}\mathit{a}\mathit{l}\mathit{c}\mathit{N}\mathit{N}\mathit{L}\mathit{S}(\overrightarrow{\mathit{v}},\widehat{\mathit{S}})$
	END For
13:	minIndex, $\mathrm{m}\mathrm{i}\mathrm{n}\mathrm{V}\mathrm{a}\mathrm{l}\mathrm{u}\mathrm{e}=\mathrm{m}\mathrm{i}\mathrm{n}(ϵ)$	// find the signature set with least relative error
14:	If $(\mathrm{m}\mathrm{i}\mathrm{n}\mathrm{V}\mathrm{a}\mathrm{l}\mathrm{u}\mathrm{e}-{ϵ}_{\mathrm{m}\mathrm{i}\mathrm{n}}\le 0.01)$
15:	$\mathit{S}=\mathit{S}[:,-\mathrm{m}\mathrm{i}\mathrm{n}\mathrm{I}\mathrm{n}\mathrm{d}\mathrm{e}\mathrm{x}]$
	else
16:	Return minValue, $\mathit{S}$
	END If
	END While
	END removeSignatures
17:	FUNCTION addSignatures $(\overrightarrow{\mathit{v}},{\mathit{S}}^{\text{all}},\mathit{S},{ϵ}_{\mathrm{m}\mathrm{i}\mathrm{n}})$
18:	While FLAG = True:
19:	For $\mathit{p}$ in 1 to size(${\mathit{S}}^{\text{all}}$, 2) do	// loop from 1 to the total number of signatures in ${\mathit{S}}^{\text{all}}$
20:	$\widehat{\mathit{S}}=\left[\mathit{S};{\mathit{S}}^{\text{all}}[:,\mathit{p}]\right]$	// add the pth signature from ${\mathit{S}}^{\text{all}}$
21:	$ϵ\left[j\right]$, ${\overrightarrow{\mathit{a}}}_{\mathit{j}}=\mathit{c}\mathit{a}\mathit{l}\mathit{c}\mathit{N}\mathit{N}\mathit{L}\mathit{S}(\overrightarrow{\mathit{v}},\widehat{\mathit{S}})$
	END For
22:	minIndex, $\mathrm{m}\mathrm{i}\mathrm{n}\mathrm{V}\mathrm{a}\mathrm{l}\mathrm{u}\mathrm{e}=\mathrm{m}\mathrm{i}\mathrm{n}(ϵ)$	// find the signature set with least relative error
23:	If $\left({ϵ}_{min}-\mathrm{m}\mathrm{i}\mathrm{n}\mathrm{V}\mathrm{a}\mathrm{l}\mathrm{u}\mathrm{e}\ge 0.05\right)$
24:	$\mathit{S}=\left[\mathit{S};{\mathit{S}}^{\text{all}}[:,\text{minValue}]\right]$
	else
25:	Return minValue, $\mathit{S}$
	END If
	END While
	END addSignatures
26:	FUNCTION calcNNLS $(\overrightarrow{\mathit{v}},\mathit{S})$
27:	$\overrightarrow{\mathit{a}}=nnls(\mathit{S},\overrightarrow{\mathit{v}})$	// Calculating NNLS with the Lawson-Hanson method
28:	$ϵ=\parallel \overrightarrow{\mathit{v}}-\mathit{S}\overrightarrow{\mathit{a}}{\parallel }_2^2/\parallel \overrightarrow{\mathit{v}}{\parallel }_2^2$	// Computing relative error
29:	Return $ϵ$, $\overrightarrow{\mathit{a}}$
	END calcNNLS

ONLINE METHODS

Distribution and Usage

SigProfilerAssignment is distributed as a Python and R package (https://github.com/AlexandrovLab/SigProfilerAssignment and https://github.com/AlexandrovLab/SigProfilerAssignmentR) with support for most operating systems and an extensive documentation at https://osf.io/mz79v/wiki/home/?view_only=5998aee28bd14ef6bda9189184a984bd. In addition, a user-friendly online interface is provided as part of the COSMIC Mutational Signatures website¹⁰ at https://cancer.sanger.ac.uk/signatures/assignment/. For compliance with EU and UK specific privacy regulations, the COSMIC website requires free registration prior to using SigProfilerAssignment. This ensures that all uploaded user data are maintained privately and purged properly.

Input data for both desktop and online versions can be provided by mutation calling and segmentation files, depending on the variant class, and is processed internally by SigProfilerMatrixGenerator^22,27. The tool supports common formats for SBS, DBS, and ID somatic mutations, including the Variant Call Format (VCF), the Mutation Annotation Format (MAF), and simple text files. Multi-sample segmentation files obtained from ASCAT²⁸, ABSOLUTE²⁹, Sequenza³⁰, FACETS³¹, Battenberg²⁸, or PURPLE³² are supported for analysis of copy number signatures. In addition, SigProfilerAssignment can use standard mutational matrices, where rows correspond to mutational channels and columns to samples, extracted from the SigProfiler suite of tools^16,22,33. Different sequencing assays (whole genome sequencing, whole exome sequencing, and targeted sequencing), species (human, mouse, and rat) , genome builds (GRCh37/38, mm9/10, and rn6), and signatures (default COSMICv3.3¹⁰, prior COSMIC versions, and custom signature databases) are supported.

The main output of SigProfilerAssignment includes the activity of each known mutational signature for each of the supplied samples, the reconstruction of the original dataset, and the probability of each individual mutation being caused by a specific signature. The latter is not provided when the input file is a mutational vector or mutational matrix as this input format lacks information about individual somatic mutations. Signature activities correspond to the specific numbers of mutations from the original catalog caused by a particular mutational process. Considering these activities, as well as the provided set of known mutational signatures, a reconstruction of the original mutational catalog for each sample is derived. Different accuracy metrics for this reconstruction are outputted by SigProfilerAssignment, including cosine similarity, Kullback–Leibler divergence, Pearson correlation, L1 relative error, and L2 relative error.

The signature assignment results are summarized using three independent visualizations: (i) a bar plot depicting the activities of all mutational signatures within a sample; (ii) a tumor mutational burden (TMB) signature plot showing the activities per mutational signature; and (iii) an individual reconstruction plot per sample, which includes the mutational profiles for both the original and the reconstructed input sample, different accuracy metrics, and the mutational profiles for each of the known mutational signatures assigned to that sample. For the online version of the tool, an interactive heatmap plot, including the signatures’ activities and the samples’ reconstruction accuracies is also provided. Raw data files containing activities, reconstruction metrics, and signature probabilities for individual mutations are generated by the desktop tool and can be downloaded from the online version.

Benchmarking of bioinformatics tools for refitting known mutational signatures

To evaluate the performance of tools for refitting known mutational signatures, we used a standard set of evaluation metrics and compared SigProfilerAssignment with another four commonly used approaches: deconstructSigs¹⁷, MutationalPatterns^18,19, sigLASSO²⁰, and SignatureToolsLib^5,21. Specifically, each tool was applied to 2,700 previously simulated cancer genomes¹⁶, corresponding to 300 simulated tumors from nine different cancer types, including: bladder transitional cell carcinoma, esophageal adenocarcinoma, breast adenocarcinoma, lung squamous cell carcinoma, renal cell carcinoma, ovarian adenocarcinoma, osteosarcoma, cervical adenocarcinoma, and stomach adenocarcinoma. The cancer genomes of these samples were simulated using 21 different COSMIC reference signatures. To emulate a typical refitting of mutational signatures, each tool was applied by utilizing the complete set of 79 COSMICv3.3 SBS signatures. After assigning the signatures, the assignment of each signature to each sample was classified as either a true positive (TP), false positive (FP), or false negative (FN) result. A known signature was considered TP if at least one mutation was assigned to the signature by a particular tool and the ground truth activity of the signature was greater than zero. In contrast, a signature was classified as FP when it was assigned by a tool, but the ground truth activity was zero. Lastly, FN results were signatures with ground truth activities above zero that were not assigned any somatic mutation. These standard metrics allowed calculating the precision, sensitivity, and F₁ score of each tool per sample, defined as:

$$\mathit{Precision}=\frac{TP}{TP+FP}$$

$$\mathit{Sensitivity}=\frac{TP}{TP+FN}$$

$$F_1\mathit{score}=2\mathrm{*}\frac{\mathit{Precision}\mathrm{*}\mathit{Sensitivity}}{\mathit{Precision}+\mathit{Sensitivity}}$$

These metrics were calculated for each synthetically generated sample and, subsequently, averaged to obtain a final accuracy value for each random noise level (0%, 5%, and 10%). To benchmark the computational performance of the different bioinformatics tools, their CPU elapsed time and peak memory usage were monitored and averaged for the three noise levels.

SigProfilerAssignment v0.0.28 was run using default parameters. deconstructSigs¹⁷ v1.8.0 was used with default parameters as indicated in https://github.com/raerose01/deconstructSigs/. MutationalPatterns¹⁸ v3.0.1 was run with default parameters independently using its standard and strict modes, corresponding to the fit_to_signatures and fit_to_signatures_strict functions, respectively. The max_delta parameter was fixed to a default value of 0.004 for the strict mode, according to authors’ instructions at https://bioconductor.org/packages/release/bioc/vignettes/MutationalPatterns/inst/doc/Introductio n_to_MutationalPatterns.html. sigLASSO²⁰ v1.1 was used with default parameters (no priors) following the instructions at https://github.com/gersteinlab/siglasso; albeit avoiding the generation of plots for the comparison of the computational performance. SignatureToolsLib⁵ v2.1.2 was run with global signatures using the Fit function and default parameters as indicated at https://github.com/Nik-Zainal-Group/signature.tools.lib.

DATA AVAILABILITY

All synthetic benchmarking data used in this article are available on FigShare at https://doi.org/10.6084/m9.figshare.20409430 and were originally generated as part of Ref. ¹⁶. They are publicly available under the Creative Commons Attribution 4.0 International license.