Table 1.
Summary of Relevant Computational Pathology Work in Musculoskeletal Research
| Tissue Type | Author/Year | Aim/Objective | Species | Stain | Imaging Modality | ML/feature Extraction Type | Technique/Model | Transfer Learning | Biological Specimens | Images (N) | Magnification | Performance Reported |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Synovial Tissues | Kraan 2000 [39] | Quantification of CD3 and CD68+ cells | Human | IHC/DAB | Microscope, Camera | Knowledge Driven | Thresholding | No | 9 RA, 5 Control subjects | 70 (n=5/section) | 40× | DIA Significantly correlated with manual cell counts, Spearman ρ: 0.56–0.95 |
| Haringman 2005 [59] | Quantification of CD68+ cells | Human | IHC/DAB | Microscope, Camera | Knowledge Driven | Thresholding | No | 88 subjects (n=176 samples) | NR | 40x | Validated in [39] | |
| Rooney 2007 [60] | Quantification of CD3 and CD68+ cells | Human | IHC/DAB | Microscope, Camera | Knowledge Driven | Thresholding | No | 12 subjects (n≥6 samples/subject, n > 72) | 24 tissue sections (n=12 slides) | 1392×1040 pixel/image | ICC across sites: CD3+ 0.79; CD68+ 0.58; Spearman ρ Manual counts vs DIA : 0.62–0.98 | |
| Morawietz 2008 [61] | Quantification of synovial features to validate synovitis score (enlargement of synovial lining (thickness), density of synovial stroma and inflammatory infiltrate (count)) | Human | H&E | Microscope, Camera | Knowledge Driven | Thresholding | No | 71 subjects (OA, n=22, PsA, n=7, RA, n=35, control, n=7) | NR |
NR 584×720 pixel/image |
Significant agreement in all measurements between the model and three independent pathology graders, Spearman ρ: 0.458–0.921 | |
| Bell 2019 [62, 63] | Nuclear and cytoplasmic/ECM area | Mouse | H&E | Slide Scanner | Supervised, Knowledge Driven | Bayesian Classifier | No | NA | NA | 40× | Previously Validated [75] | |
| Venerito 2021 [43] | Quantification and classification of synovitis | Human | H&E | Microscope, Camera | Supervised, Data Driven | CNN/Resnet34 | Yes | 12 subjects | 150 | 4–20× |
Validation Set - Acc.: 0.9; Prec.: 0.93; Rec.: 0.875 Test Set – Acc.: 1.0; Prec.: 1.0, Rec.: 1.0 |
|
| Cartilage | Knight 2001 [64] | Vimentin and microtubule spatial organization | Bovine | IHC-IF | Confocal | Knowledge driven | Convolutional Filters | No | NR | NR | 60× | Not validated |
| Moussavi-Harami 2009 [65] | Automated and Objective implementation of the Mankin Scoring Scale | Human | Safranin-O | Microscope, Camera | Knowledge Driven | Custom Features Extraction | No | 18 subjects (femoral heads, n =12, femoral condyles, n=5, tibial plateau, n=7) | NR |
4× stitched (743,028 pixels/mm2 resolution) |
Correlated well with Manikin Scoring (r 2=0.748) | |
| Yang 2019 [44] | Chondrocyte detection, count, and boundary segmentation | Rabbit | Safranin-O | Microscope, Camera | Supervised, Data Driven | CNN/U-Net | No | NR | 260 |
256×256 pixel/image 0.32 μm/pixel |
F1 scores: 0.86–0.90; segmentation accuracy: IoU=0.828; counted fewer chondrocytes than expert observer (p<0.001 paired t test) |
|
| Skeletal muscle | Klemencic 1998 [66] | Fiber Geometry | Human | Myofibrillar ATPase Activity | Microscope, Camera | Unsupervised, Knowledge Driven | Active Contour Model | NA | 1 subject | NR |
512×360 pixel/image 2.2 μm/pixel |
Qualitative 92% correct by expert graders |
| Kim 2007 [48] | Fiber geometry | Human | H&E | Microscope, Camera | Unsupervised, Knowledge Driven | Active Contour Model | NA | 5 subjects | 30 |
20× 640×480 pixel/image |
663/679 (98%) fibers correctly detected; | |
| Sertel 2011 [67] | Fiber Geometry and Type | Rat | ATPase Activity | Microscope, Camera | Unsupervised, Knowledge Driven | Ridge detection | NA | 12 subjects | 25 |
10× 1280×1024 pixel/image |
Overlap score: 91.3 ± 4.8% | |
| Liu 2013 [68] | Fiber geometry, Type, Myonuclei Counting | Mouse | IHC-IF | Microscope, Camera | Unsupervised/Supervised, Knowledge Driven | Ridge detection, SVM | NA | NR | 20 | 20× |
CSA Avg Diff: 0.88% Fiber type Avg Diff: 0.09% Nuclei counting Diff: 8.61% |
|
| Smith and Barton 2014 [69] | Fiber Geometry, Type, MHC, Capillary Density, and CNF | Mouse | IHC-IF | Microscope, Camera | Knowledge Driven | Filtering and Watershed | No | 8 subjects (n=4/group) | NR | NR |
Difference Reported to Legacy Method (Simple Thresholding) CSA: 21.7% Fiber type: 7/177 fibers CNF: 9% |
|
| Wen 2018 [49] | Fiber Geometry, Type, and Myonuclei Counting | Mouse | IHC-IF | Microscope, Camera | Semi-supervised, Knowledge Driven | Watershed with Euclidean Distance K-Means Optimization | No | 16 (n=4/group) | NR | 20× | Accuracy of ≥94% for fiber number, fiber type distribution, fiber CSA, and myonuclear number | |
| Miazaki 2015 [70] | Fiber Number and Geometry | Mouse | IHC-IF | Microscope, Camera—Stitched Into Mosaic | Unsupervised, Knowledge Driven | Filtering, Thresholding and Post-Hoc Shape Filtering | No | 6 subjects (n=3/group) |
6 20–30 stitched/sample |
800×600 pixels/image 0.7 μm/pixel |
NR | |
| Mayeuf-Louchart 2018 [71] | Fiber Number, Geometry, Type, CNF, Satellite Cells, and Vessel | Mouse | IHC-IF | Slide Scanner | Knowledge Driven | Filtering, Thresholding and Post-Hoc Shape Filtering | No | 9 subjects (n=5, injured, n=4, control) | NR |
20–40× 0.325–0.380 μm/pixel |
No significant difference between expert graders and digital analysis in both uninjured and injured for all parameters, Mann-Whitney test p value: 0.4–0.7 | |
| Reyes-Fernandez 2019 [72] | Fiber Number and Geometry | Human | IHC-IF | Microscope, Camera | Knowledge Driven | Filtering and Thresholding | No | 57 subjects | NR |
10× 9300×9900 pixels/image |
Overall detection/segmentation of 89.3% of the total fibers (342/3212 not detected fibers across 10 samples analyzed); < 1% of the fibers misclassified (21/3212) |
|
| Kastenschmidt 2019 [45] | Fiber Number, Geometry, Type, and CNF | Human and Mouse | IHC-IF | Microscope, Camera—Stitched into Mosaic | Supervised, Knowledge Driven | Filtering and Thresholding; SVM | No |
NR (Human) 108 subjects (Mouse) |
NR (Human) NR (Mouse) |
10× (Human) 20× (Mouse) 1920×1440 pixels/image (Mouse) |
Fiber number Acc.: 80–98%; CSA Acc.: 90–98%; CNF Acc.: 85–95%; Fiber Type Acc.: NR | |
| Encarnacion-Rivera 2020 [73] | Fiber Number, Geometry and Type | Mouse | IHC-IF | Microscope, Camera—Stitched into Mosaic | Knowledge Driven | Convolutional Filtering; Random Forest; Thresholding | No | 32 subjects (n=29, C57BL/6J, n=3, mdx-4Cv) |
~192 6/subject |
10× |
Count: r 2=0.99 with manual count CSA: Not Different than Manual annotation (2 annotators) Type: 1–5% False Positives |
|
| Other | Zhang 2016 [79] | Bone Fracture Healing Tissue Areas: New Cartilage, New Bone, New Fibrous Tissue, Bone Marrow and New Osteoblastic Area | Mouse | H&E – Orange G - Alcian Blue | Slide scanner | Knowledge Driven | Model Not Reported; Post-Hoc Area and Shape Adjustments | No | 5 subjects (Mouse) | 5 | 40× |
ICCs between the Algorithm and Hand Drawn Areas: New Cartilage = 0.98, New Bone = 0.99, New Fibrous Tissue = 0.97 |
| Xia 2021 [46] | Wound Healing via Area of Primary Granulation, Secondary Granulation and Chondrogenic Tissue over Time | Mouse | H&E | Slide scanner | Supervised, Knowledge Driven | Random Forest | No | 4 subjects (Mouse) | 4 | 40× | Good agreement between model and pathologist scores | |
| Correia 2020 [47] | Develop DL-based score to mimic mRSS which discriminates SSc from normal skin | Human | Masson’s Trichrome | Slide Scanner | Unsupervised, Supervised, Data Driven |
DCNN (Encoder of AlexNet); Principal Component Analysis; Logistic Regression |
Yes | 92 subjects, 168 biopsies; Primary cohort (n = 6 subjects, 26 SSc biopsies); Secondary cohort (n = 60 SSc and 16 controls, 148 biopsies) | 100 randomly selected; Primary cohort (2600 image patches grouped by biopsy); Secondary cohort (7600 image patches grouped by biopsy) | 40× |
Primary Cohort Biopsy Score Correlation with mRSS: R=0.55, p=0.01; Secondary Cohort Diagnostic Score Logistic Regression to Classify SSc from Healthy (0.5 cutoff): AUC = 0.99 Misclassification rate = 1.9% (training), 6.6% (test); Secondary Cohort Fibrosis Score significantly correlated with mRSS: R=0.70 (training), 0.55 (test) |
Abbreviations: DAB 3,3′-Diaminobenzidine, AUC area under the curve, Avg average, CNF centrally nucleated fibers, CSA cross-sectional area, DCNN deep convolutional neural network, DL deep learning, Diff difference, DIA digital image analysis, ECM extracellular matrix, H&E hematoxylin and eosin, IHC immunohistochemistry, IF immunofluorescence, IoU intersection over union, ICC intraclass correlation coefficient, ML machine learning, mRSS modified Rodnan skin score, MHC myosin heavy chain, NA not applicable, NR not reported, OA osteoarthritis, RA rheumatoid arthritis, SSc systemic sclerosis, SVM support vector machine