Figure 4.

The performance and scalability of CellAnn. (A) Evaluation framework of CellAnn. According to the author’s annotation, predictions are classified into “Correct Classified,” “Correct Classified (Partially),” “Wrong Classified,” “Failed Classified,” “Wrong Unclassified,” or “Correct Unclassified.” (B) The benchmark results of 56 testing pairs for 6 different methods. Each bar indicates the composition of predicted cell types. Based on the overlapping of cell types between query and reference datasets, we divided these test pairs into four groups: type 1, type 2, type 3, and type 4. The Venn diagrams on the left show the relationships of type 1–type 4. (C) The bar plots indicate the composition of predicted categories of the average performance in a collection of reference–testing pairs. (D) Benchmarking the efficiency of CellAnn. Left: the line plot shows the running time under the default settings of each algorithm. Right: the line plot shows the peak memory usage of each algorithm. The x-axis is labeled by the number of cells and the clusters in the references. The curves are truncated if a method is not scalable to a certain size of the references.