Table 5.
Per-pixel confusion matrix of Non-Experts to Expert Consensus.
| Expert Consensus | ||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Water | Trees | Grass | Flooded Vegetation | Crops | Shrub & Scrub | Built Area | Bare Ground | Snow & Ice | Cloud | Precision/User’s | ||
| Non-Experts | Water | 7814103 | 36668 | 21205 | 58174 | 193344 | 18921 | 17533 | 68692 | 3688 | 3002 | 94.90% |
| Trees | 237858 | 16664442 | 499046 | 301831 | 657282 | 1834363 | 133325 | 71652 | 18987 | 19774 | 81.50% | |
| Grass | 9921 | 60697 | 540597 | 55648 | 265055 | 113899 | 14984 | 18240 | 550 | 1407 | 50.00% | |
| Flooded Vegetation | 255910 | 107576 | 23721 | 879482 | 40311 | 33552 | 9357 | 23270 | 0 | 0 | 64.00% | |
| Crops | 13392 | 150401 | 258153 | 7920 | 12567377 | 223368 | 92983 | 88258 | 0 | 5141 | 93.70% | |
| Shrub & Scrub | 72746 | 1748087 | 1008622 | 519267 | 1656386 | 5143322 | 245498 | 1996929 | 137415 | 12010 | 41.00% | |
| Built Area | 10716 | 222831 | 40154 | 2028 | 324127 | 59196 | 6224787 | 35812 | 866 | 1510 | 89.90% | |
| Bare Ground | 22661 | 25522 | 23480 | 4482 | 268845 | 257830 | 22460 | 1036373 | 49328 | 4090 | 60.40% | |
| Snow & Ice | 2644 | 59334 | 539 | 687 | 73 | 21734 | 0 | 15917 | 1388854 | 11 | 93.20% | |
| Cloud | 3033 | 8988 | 198 | 264 | 19887 | 244 | 3740 | 393 | 3955 | 161082 | 79.80% | |
| Recall/Producer’s | 92.60% | 87.30% | 22.40% | 48.10% | 78.60% | 66.70% | 92.00% | 30.90% | 86.60% | 80.20% | 77.80% | |
Note that cloud is included as both sets of annotations include this label (n = 58,963,662).