Table 1.
Skin tone estimation performance across multiple machine learning (ML) models and preprocessing techniques.
| Raw Masked Pixels | Feature Vectors (HOG + ITA) | |||||||
|---|---|---|---|---|---|---|---|---|
| Model | Accuracy | F1 score | Precision | AUROC | Accuracy | F1 score | Precision | AUROC |
| Random Forest34 | 0.87 ± 0.00 | 0.83 ± 0.00 | 0.84 ± 0.00 | 0.77 ± 0.01 | 0.88 ± 0.00 | 0.85 ± 0.00 | 0.86 ± 0.00 | 0.84 ± 0.00 |
| Balanced Random Forest | 0.76 ± 0.00 | 0.79 ± 0.00 | 0.86 ± 0.00 | 0.80 ± 0.00 | 0.77 ± 0.01 | 0.80 ± 0.00 | 0.87 ± 0.00 | 0.85 ± 0.00 |
| Extremely Randomized Trees35 | 0.87 ± 0.00 | 0.83 ± 0.00 | 0.85 ± 0.00 | 0.80 ± 0.01 | 0.88 ± 0.00 | 0.85 ± 0.00 | 0.86 ± 0.01 | 0.85 ± 0.01 |
| Ada Boost36 | 0.85 ± 0.00 | 0.84 ± 0.00 | 0.82 ± 0.00 | 0.77 ± 0.00 | 0.87 ± 0.00 | 0.86 ± 0.00 | 0.85 ± 0.00 | 0.83 ± 0.01 |
| Gradient Boosting | 0.86 ± 0.00 | 0.86 ± 0.00 | 0.84 ± 0.00 | 0.77 ± 0.02 | 0.88 ± 0.00 | 0.86 ± 0.00 | 0.86 ± 0.00 | 0.85 ± 0.01 |
| Pretrained Resnet (STAR-ED) | 0.90 ± 0.00 | 0.91 ± 0.00 | 0.91 ± 0.00 | 0.87 ± 0.01 | NA | NA | NA | NA |
The metrics are based on cross-validation for five stratified folds over the Fitzpatrick17K dataset. Different validations include using Raw Masked Pixels without handcrafting features to the ML models. In another validation, features were manually extracted and fed into the ML models. These features include Histogram of Oriented Gradient (HOG), which is a commonly employed and simple image representation, and an Individual Topology Angle (ITA) that is used to map skin images into Fitzpatrick skin tone categories. Expectedly, traditional models such as Ada boost and Random Forest performed better using handcrafted features, whereas a pretrained ResNet finetuned with Fitzpatrick1711 exploited the raw masked pixels due to its capability to learn discriminant features automatically and achieved the highest performance by outperforming all the baseline models.
Bold entries represent the best performing method.