Table 1.
Overview of use cases and measured efficiency improvements
| Level of cellular and structural complexity | Histological primitive | Number of ROI | Number of histological objects labeled | PS total time (s) | PS human time (s) | PS efficiency total time (LPS) | PS efficiency human time (LPS) | Manual efficiency (LPS) | Speed up (θ) | PS efficiency lower bound (LPS) with speed up | PS efficiency upper bound (LPS) with speed up | Stain type | Concordance |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Low | Breast cancer nuclei: lymphocytes vs. non-lymphocytes | 2000 | 101479 | 235998 | 32735 | 1.92 | 3.1 | 0.43 | 7.21x | 0.35 (0.81x) | 9.6 (22.3x) | H&E | 86% |
| Medium | Lung cancer tumor-budding: present vs. absent | 27 | 1631 | 2471 | 1800 | 0.66 | 0.906 | 0.292 | 3.1x | 0.49 (1.68x) | 1.06 (3.6x) | H&E | 93% |
| Medium | Kidney tubules: distal vs. proximal vs. abnormal | 216 | 2298 | 10943 | 3648 | 0.21 | 0.63 | 0.218 | 2.89x | 0.52 (2.47x) | 0.95 (4.5x) | PAS | 97% |
| High | Kidney glomeruli: SS vs. GS vs non-SS/GS | 16158 | 16158 | 23978 | 20171 | 0.674 | 0.801 | 0.159 | 5.03x | 0.57 (3.58x) | 1.15 (7.23x) | PAS | 96% |
Description of the datasets used for validating PatchSorter along with the demonstrated efficiency gains in terms of labels per second (LPS) and concordance with an unaided approach. The difference between human time and total time is the inclusion of model training and embedding in the lableling time in total time, while it is removed for human time, as the human reader can be dismissed to perform other non-labeling related tasks. Manual efficiency (LPSM) for the same task is estimated based on the extrapolatation of manual labeling of a subset of the data within a 15-min interval. Upper and lower bounds for the efficiency of PS (LPSPS) are estimated within a 15-min interval sliding window with a 5-min interval stride (see Fig. 2). This creates robust estimates of the upper and lower bounds by smoothing potential outliers, as well as allowing for more accurate comparisons to LPSM. For the nuclei use case, speed increases of up to 22.3x (9.6 LPS) are observable while only being slightly slower than manual labeling in one of the measured 15-min intervals. For tubules, tumor buds, and glomeruli, PatchSorter offers a speed increase over manual labeling efforts, even for worst-case estimates.
SS segmentally sclerotic, GS globally sclerotic, non-SS/GS non-sclerotic.