TABLE I. Synthetic Data Can Effectively be Used in Place of Real Data: Median Metrics Comparing Training With Real Datasets to Training With Synthetic Datasets. The First Section Compares Inter-Observer Results for on Real and Synthetic Data. The Next Section Shows Networks Trained on Real and Synthetic Data for LVendo Segmentation in A4C ED Images and Tested on EchoNet. The Final Section Compares Networks Trained on Real and Synthetic Data for All Annotations/Views/Phases. All Dice Results Are Statistically Different With a p-Value < 0.05 (Computed With a Wilcoxon Signed-Rank Test) Except for the Marked Comparison (†). Results are Ordered by Dice Score and Bold Shows the Best Result in Each Section.
Task | Training Data | Testing Data | (%) [] | (%) | ||||||
---|---|---|---|---|---|---|---|---|---|---|
OR inter-observer comparison | LVendo | LVepi | LA | LVendo | LVepi | LA | LVendo | |||
Inter- | vs. Camus/SiteA | 87.9 [4.4] | 93.4 [2.2] | 87.5 [6.1] | 24 | 8 | −3 | 0.83 | 5.0 | |
Observer | vs. proposed pipeline | 90.8 [2.3] | 92.0 [2.1] | 83.0 [8.0] | 9 | 4 | 14 | 0.80 | 3.9 | |
LVendo in A4C ED | EchoNet (base) | EchoNet | 94.0 [1.6] | n/a | n/a | 0 | n/a | n/a | 0.85 | 2.7 |
Camus | 89.6 [2.6] | −2 | 0.87 | 4.8 | ||||||
SiteA | 87.7 [3.8] | 14 | 0.86 | 5.6 | ||||||
Synth EchoNet | 87.1 [3.7] | 15 | 0.85 | 5.9 | ||||||
All | Camus (base) | Camus | 93.3 [2.3] | 95.7 [1.0] | 92.0 [3.2] | 1 | 0 | 1 | 0.84 | 2.6 |
SiteA | 88.9 [3.4] | 92.3 [2.1] | 85.4 [5.3] | 10 | −2 | 14 | 0.85 | 4.8 | ||
Synth Camus | 81.2 [5.4] | 90.3 [2.8] | 79.6 [8.3] | 33 | 7 | −5 | 0.83 | 7.5 | ||
Synth SiteA | SiteB | 88.4 [3.8] | 90.7 [3.4] | 83.1 [9.4] | −9 | −1 | −11 | 0.83 | 5.4 | |
SiteA (base) | 85.1 [4.0] | 91.3 [2.0]† | 84.4 [4.7] | −27 | −8 | 8 | 0.84 | 7.5 | ||
Camus | 81.8 [4.9] | 91.9 [1.8]† | 86.6 [4.4] | −35 | −10 | −10 | 0.83 | 8.7 | ||
Camus | Pathological | 89.8 [2.6] | 92.5 [1.8] | 92.2 [2.0] | −9 | −2 | 8 | 0.86 | 4.1 | |
SiteA (base) | 89.0 [3.3] | 92.0 [2.9] | 87.5 [3.1] | −4 | 5 | 19 | 0.85 | 4.2 | ||
Synth SiteA | 88.3 [4.9] | 87.6 [4.1] | 84.3 [6.1] | 9 | 5 | −2 | 0.84 | 4.6 |
LVendo = left ventricle endocardium, LVepi = left ventricle epicardium, LA = left atrium, A4C = apical four chamber, A2C = apical two chamber, ED = end diastole, ES = end systole. All refers to all annotations (LVendo, LVepi, and LA), views (A4C and A2C), and phases (ED and ES). = Dice score, = median absolute deviation, = Bias percentage, = simplicity, and = mean average distance. For inter-observer is listed for the second round annotations and is calculated as Original.
Not statistically different with a P-value < 0.05.