Extended Data Fig. 4. MuSiCal outperforms SigProfilerExtractor and three additional existing tools – signature.tools.lib, SignatureAnalyzer, and SigneR – for de novo signature discovery at different noise levels.
a. Area under precision-recall curve (auPRC) for all five tools at different noise levels. Random noise were added as in Extended Data Fig. 3. Each box in the box plot represents 250 synthetic datasets (25 tumor types × 10 replicates). auPRC was calculated for each dataset separately, as in Fig. 3b. ***: p < 0.0005. p-values were calculated with two-sided paired t-tests. Raw p-values from top to bottom at noise level 0.0: 1.2 × 10−26, 1.4 × 10−15, 1.1 × 10−14, 7.5 × 10−10. Raw p-values from top to bottom at noise level 0.05: 1.7 × 10−25, 1.3 × 10−20, 5.8 × 10−16, 3.1 × 10−11. b. Precision-recall curve (PRC) for all five tools at different noise levels. Each PRC represents the average result of 250 synthetic datasets (25 tumor types × 10 replicates), as in Fig. 3c. c. Precision of all five tools averaged across all tumor types at different noise levels. Recall was fixed at 0.9. Error bars indicate standard deviation over 10 replicates. d. Recall of all five tools averaged across all tumor types at different noise levels. Precision was fixed at 0.9, corresponding to a false discovery rate (FDR) of 10%. Here, precision was fixed at a smaller value compared to Fig. 3e, Supplementary Fig. 5d, and Extended Data Fig. 3d, because a precision of 0.98 was never achieved by SignatureAnalyzer or SigneR in many cases. Error bars indicate standard deviation over 10 replicates.