Table 1.
Efficiency and risk of automated label cleaning in color fundus photograph
| NR | Six-Iteration strategy | DQS-guided strategy | ||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| ACC | CMR | EMR | MR | FLR | ACC | CMR | EMR | MR | FLR | |
| 0% | 0.994 ± 0.001 | - | - | - | 0.7% ± 0.7% | 1.000 ± 0.000 | - | - | - | 0.0% ± 0.0% |
| 5% | 0.991 ± 0.002 | 97.5% ± 1.3% | 0.4% ± 0.7% | 2.1% ± 0.7% | 0.9% ± 0.1% | 0.993 ± 0.002 | 96.7% ± 1.4% | 0.4% ± 0.7% | 2.9% ± 0.7% | 0.6% ± 0.1% |
| 10% | 0.983 ± 0.003 | 96.7% ± 2.0% | 1.5% ± 2.0% | 1.9% ± 0.0% | 1.6% ± 0.5% | 0.983 ± 0.002 | 95.2% ± 1.0% | 1.5% ± 1.6% | 3.3% ± 0.7% | 1.4% ± 0.1% |
| 15% | 0.978 ± 0.009 | 95.8% ± 0.8% | 2.4% ± 0.6% | 1.8% ± 0.6% | 1.9% ± 1.0% | 0.977 ± 0.007 | 94.3% ± 2.1% | 2.6% ± 0.9% | 3.1% ± 1.3% | 1.7% ± 0.8% |
| 20% | 0.976 ± 0.002 | 96.3% ± 1.3% | 2.0% ± 0.9% | 1.8% ± 0.7% | 2.1% ± 0.4% | 0.977 ± 0.001 | 96.0% ± 1.1% | 2.1% ± 1.0% | 1.9% ± 0.6% | 2.0% ± 0.3% |
| 25% | 0.966 ± 0.003 | 95.3% ± 1.6% | 2.7% ± 0.5% | 2.0% ± 1.1% | 2.8% ± 0.3% | 0.969 ± 0.001 | 94.9% ± 1.5% | 2.8% ± 0.6% | 2.3% ± 0.9% | 2.4% ± 0.4% |
| 30% | 0.970 ± 0.006 | 96.0% ± 1.4% | 2.6% ± 0.8% | 1.4% ± 0.6% | 2.6% ± 0.2% | 0.970 ± 0.003 | 95.3% ± 1.4% | 2.6% ± 0.5% | 2.1% ± 0.9% | 2.4% ± 0.2% |
| 35% | 0.964 ± 0.005 | 95.2% ± 0.6% | 3.2% ± 0.4% | 1.6% ± 0.2% | 2.8% ± 0.1% | 0.964 ± 0.003 | 94.6% ± 0.3% | 3.2% ± 0.4% | 2.3% ± 0.3% | 2.7% ± 0.2% |
| 40% | 0.961 ± 0.009 | 95.1% ± 0.9% | 3.2% ± 0.5% | 1.7% ± 0.5% | 3.3% ± 0.9% | 0.959 ± 0.009 | 94.7% ± 0.9% | 3.3% ± 0.5% | 1.9% ± 0.5% | 3.3% ± 0.9% |
| 45% | 0.959 ± 0.001 | 95.1% ± 0.3% | 3.1% ± 0.5% | 1.8% ± 0.3% | 3.4% ± 0.1% | 0.957 ± 0.002 | 94.4% ± 0.4% | 3.4% ± 0.5% | 2.2% ± 0.5% | 3.2% ± 0.0% |
| 50% | 0.958 ± 0.003 | 95.1% ± 0.4% | 3.5% ± 0.3% | 1.4% ± 0.3% | 3.5% ± 0.3% | 0.956 ± 0.003 | 94.6% ± 0.7% | 3.8% ± 0.6% | 1.6% ± 0.3% | 3.6% ± 0.2% |
| 55% | 0.948 ± 0.004 | 93.9% ± 0.4% | 4.2% ± 0.4% | 1.9% ± 0.6% | 4.2% ± 0.3% | 0.946 ± 0.005 | 93.5% ± 0.7% | 4.4% ± 0.3% | 2.1% ± 0.7% | 4.1% ± 0.3% |
| 60% | 0.950 ± 0.006 | 94.4% ± 0.7% | 4.0% ± 0.4% | 1.5% ± 0.4% | 4.1% ± 0.8% | 0.930 ± 0.016 | 94.1% ± 1.1% | 4.2% ± 0.6% | 1.8% ± 0.5% | 4.2% ± 0.7% |
| 65% | 0.942 ± 0.010 | 93.3% ± 1.5% | 4.9% ± 1.4% | 1.8% ± 0.2% | 4.2% ± 1.1% | 0.932 ± 0.020 | 92.2% ± 2.5% | 5.4% ± 1.7% | 2.4% ± 0.9% | 4.9% ± 1.3% |
| 70% | 0.929 ± 0.004 | 92.3% ± 0.6% | 5.9% ± 0.2% | 1.8% ± 0.5% | 5.6% ± 0.8% | 0.929 ± 0.005 | 92.3% ± 0.7% | 5.9% ± 0.3% | 1.8% ± 0.5% | 5.7% ± 0.8% |
NR noise rate, ACC label accuracy, CMR correct modification rate, EMR error modification rate, MR miss rate, FLR false labeling rate.