Table 2.
The detailed results of different classifiers among various deep models for predicting pathologic response following NAC in breast cancer patients.
| Deep learning model | Model | AUC | 95% CI | Sensitivity | Specificity | PPV | NPV | |
|---|---|---|---|---|---|---|---|---|
| ViT | SVM | train | 0.90 | 0.86-0.94 | 0.69 | 0.92 | 0.88 | 0.78 |
| test | 0.73 | 0.61-0.86 | 0.54 | 0.77 | 0.65 | 0.67 | ||
| KNN | train | 0.77 | 0.71-0.82 | 0.63 | 0.76 | 0.69 | 0.71 | |
| test | 0.63 | 0.49-0.76 | 0.54 | 0.59 | 0.52 | 0.61 | ||
| RandomForest | train | 1.00 | 1.00-1.00 | 0.95 | 1.00 | 1.00 | 0.96 | |
| test | 0.74 | 0.61-0.87 | 0.50 | 0.82 | 0.70 | 0.66 | ||
| ExtraTrees | train | 1.00 | 1.00-1.00 | 1.00 | 1.00 | 1.00 | 1.00 | |
| test | 0.59 | 0.45-0.74 | 0.32 | 0.71 | 0.47 | 0.56 | ||
| XGBoost | train | 0.99 | 1.00-1.00 | 0.98 | 0.985 | 0.983 | 0.985 | |
| test | 0.72 | 0.59-0.85 | 0.61 | 0.67 | 0.61 | 0.67 | ||
| LightGBM | train | 0.93 | 0.89-0.96 | 0.80 | 0.89 | 0.86 | 0.84 | |
| test | 0.74 | 0.61-0.87 | 0.57 | 0.73 | 0.64 | 0.676 | ||
| MLP | train | 0.80 | 0.74-0.85 | 0.59 | 0.86 | 0.78 | 0.71 | |
| test | 0.78 | 0.67-0.89 | 0.50 | 0.79 | 0.67 | 0.66 | ||
| VGG16 | SVM | train | 0.92 | 0.88-0.95 | 0.76 | 0.89 | 0.85 | 0.81 |
| test | 0.76 | 0.63-0.90 | 0.82 | 0.77 | 0.74 | 0.84 | ||
| KNN | train | 0.82 | 0.77-0.86 | 0.65 | 0.85 | 0.78 | 0.74 | |
| test | 0.70 | 0.57-0.83 | 0.68 | 0.71 | 0.66 | 0.73 | ||
| RandomForest | train | 1.00 | 1.00-1.00 | 0.96 | 1.00 | 1.00 | 0.97 | |
| test | 0.67 | 0.54-0.81 | 0.50 | 0.71 | 0.58 | 0.63 | ||
| ExtraTrees | train | 1.00 | 1.00-1.00 | 1.00 | 1.00 | 1.00 | 1.00 | |
| test | 0.64 | 0.50-0.78 | 0.57 | 0.74 | 0.64 | 0.68 | ||
| XGBoost | train | 1.00 | 1.00-1.00 | 0.99 | 1.00 | 1.00 | 0.99 | |
| test | 0.71 | 0.57-0.84 | 0.57 | 0.74 | 0.64 | 0.68 | ||
| LightGBM | train | 0.97 | 0.95-0.99 | 0.85 | 0.95 | 0.93 | 0.88 | |
| test | 0.67 | 0.54-0.81 | 0.57 | 0.62 | 0.55 | 0.64 | ||
| MLP | train | 0.85 | 0.80-0.90 | 0.66 | 0.82 | 0.76 | 0.74 | |
| test | 0.79 | 0.67-0.90 | 0.75 | 0.79 | 0.75 | 0.79 | ||
| ShuffleNet_v2 | SVM | train | 0.92 | 0.89-0.96 | 0.79 | 0.92 | 0.89 | 0.84 |
| test | 0.81 | 0.70-0.92 | 0.55 | 0.91 | 0.84 | 0.71 | ||
| KNN | train | 0.80 | 0.74-0.85 | 0.75 | 0.73 | 0.70 | 0.77 | |
| test | 0.70 | 0.57-0.83 | 0.62 | 0.71 | 0.64 | 0.69 | ||
| RandomForest | train | 1.00 | 1.00-1.00 | 0.94 | 1.00 | 1.00 | 0.95 | |
| test | 0.65 | 0.51-0.79 | 0.45 | 0.82 | 0.68 | 0.64 | ||
| ExtraTrees | train | 1.00 | 1.00-1.00 | 1.00 | 1.00 | 1.00 | 1.00 | |
| test | 0.67 | 0.54-0.81 | 0.45 | 0.79 | 0.65 | 0.63 | ||
| XGBoost | train | 1.00 | 1.00-1.00 | 1.00 | 1.00 | 1.00 | 1.00 | |
| test | 0.77 | 0.65-0.89 | 0.55 | 0.85 | 0.76 | 0.69 | ||
| LightGBM | train | 0.94 | 0.92-0.97 | 0.80 | 0.94 | 0.92 | 0.85 | |
| test | 0.74 | 0.61-0.86 | 0.41 | 0.82 | 0.67 | 0.62 | ||
| MLP | train | 0.84 | 0.80-0.89 | 0.67 | 0.90 | 0.85 | 0.76 | |
| test | 0.81 | 0.69-0.92 | 0.45 | 0.97 | 0.93 | 0.67 | ||
| ResNet18 | SVM | train | 0.96 | 0.94-0.98 | 0.88 | 0.94 | 0.93 | 0.90 |
| test | 0.81 | 0.70-0.92 | 0.72 | 0.82 | 0.78 | 0.78 | ||
| KNN | train | 0.86 | 0.82-0.90 | 0.62 | 0.91 | 0.86 | 0.74 | |
| test | 0.64 | 0.51-0.78 | 0.52 | 0.68 | 0.58 | 0.62 | ||
| RandomForest | train | 1.00 | 0.99-1.00 | 0.97 | 0.99 | 0.98 | 0.98 | |
| test | 0.61 | 0.48-0.75 | 0.45 | 0.65 | 0.52 | 0.58 | ||
| ExtraTrees | train | 1.00 | 1.00-1.00 | 1.00 | 1.00 | 1.00 | 1.00 | |
| test | 0.61 | 0.47-0.75 | 0.45 | 0.79 | 0.65 | 0.63 | ||
| XGBoost | train | 1.00 | 1.00-1.00 | 1.00 | 1.00 | 1.00 | 1.00 | |
| test | 0.59 | 0.44-0.73 | 0.69 | 0.47 | 0.53 | 0.64 | ||
| LightGBM | train | 0.96 | 0.94-0.98 | 0.80 | 0.96 | 0.94 | 0.85 | |
| test | 0.59 | 0.44-0.73 | 0.31 | 0.59 | 0.39 | 0.50 | ||
| MLP | train | 0.87 | 0.82-0.91 | 0.77 | 0.81 | 0.77 | 0.80 | |
| test | 0.87 | 0.78-0.96 | 0.83 | 0.74 | 0.73 | 0.83 | ||
| MobileNet_v2 | SVM | train | 0.91 | 0.87-0.95 | 0.76 | 0.90 | 0.87 | 0.82 |
| test | 0.72 | 0.59-0.85 | 0.52 | 0.91 | 0.83 | 0.69 | ||
| KNN | train | 0.78 | 0.72-0.83 | 0.69 | 0.75 | 0.70 | 0.74 | |
| test | 0.59 | 0.46-0.73 | 0.35 | 0.65 | 0.46 | 0.54 | ||
| RandomForest | train | 1.00 | 1.00-1.00 | 0.97 | 0.99 | 0.99 | 0.98 | |
| test | 0.63 | 0.49-0.76 | 0.35 | 0.77 | 0.56 | 0.58 | ||
| ExtraTrees | train | 1.00 | 1.00-1.00 | 1.00 | 1.00 | 1.00 | 1.00 | |
| test | 0.57 | 0.42-0.71 | 0.41 | 0.65 | 0.50 | 0.56 | ||
| XGBoost | train | 1.00 | 1.00-1.00 | 1.00 | 1.00 | 1.00 | 1.00 | |
| test | 0.70 | 0.57-0.83 | 0.48 | 0.74 | 0.61 | 0.63 | ||
| LightGBM | train | 0.94 | 0.91-0.97 | 0.83 | 0.94 | 0.92 | 0.87 | |
| test | 0.70 | 0.57-0.83 | 0.45 | 0.88 | 0.77 | 0.65 | ||
| MLP | train | 0.83 | 0.78-0.88 | 0.63 | 0.85 | 0.78 | 0.73 | |
| test | 0.74 | 0.62-0.87 | 0.62 | 0.79 | 0.72 | 0.71 | ||
| MnasNet-0.5 | SVM | train | 0.87 | 0.83-0.92 | 0.61 | 0.92 | 0.86 | 0.74 |
| test | 0.65 | 0.52-0.79 | 0.38 | 0.74 | 0.55 | 0.58 | ||
| KNN | train | 0.77 | 0.71-0.83 | 0.66 | 0.76 | 0.70 | 0.73 | |
| test | 0.55 | 0.41-0.69 | 0.41 | 0.68 | 0.52 | 0.58 | ||
| RandomForest | train | 1.00 | 1.00-1.00 | 0.97 | 0.99 | 0.99 | 0.97 | |
| test | 0.58 | 0.44-0.72 | 0.41 | 0.65 | 0.50 | 0.56 | ||
| ExtraTrees | train | 1.00 | 1.00-1.00 | 1.00 | 1.00 | 1.00 | 1.00 | |
| test | 0.68 | 0.55-0.81 | 0.45 | 0.74 | 0.59 | 0.61 | ||
| XGBoost | train | 1.00 | 1.00-1.00 | 1.00 | 0.99 | 0.99 | 1.00 | |
| test | 0.66 | 0.51-0.80 | 0.55 | 0.79 | 0.70 | 0.68 | ||
| LightGBM | train | 0.92 | 0.89-0.96 | 0.75 | 0.91 | 0.88 | 0.81 | |
| test | 0.75 | 0.63-0.88 | 0.41 | 0.88 | 0.75 | 0.64 | ||
| MLP | train | 0.79 | 0.73-0.84 | 0.54 | 0.85 | 0.76 | 0.69 | |
| test | 0.65 | 0.50-0.79 | 0.35 | 0.65 | 0.46 | 0.54 | ||
| GoogleNet | SVM | train | 0.93 | 0.90-0.96 | 0.74 | 0.91 | 0.88 | 0.81 |
| test | 0.80 | 0.68-0.91 | 0.62 | 0.77 | 0.69 | 0.70 | ||
| KNN | train | 0.80 | 0.74-0.85 | 0.69 | 0.80 | 0.75 | 0.75 | |
| test | 0.66 | 0.53-0.80 | 0.59 | 0.77 | 0.68 | 0.68 | ||
| RandomForest | train | 1.00 | 0.99-1.00 | 0.96 | 0.99 | 0.98 | 0.96 | |
| test | 0.69 | 0.56-0.82 | 0.52 | 0.74 | 0.63 | 0.64 | ||
| ExtraTrees | train | 1.00 | 1.00-1.00 | 1.00 | 1.00 | 1.00 | 1.00 | |
| test | 0.65 | 0.51-0.78 | 0.55 | 0.65 | 0.57 | 0.63 | ||
| XGBoost | train | 1.00 | 1.00-1.00 | 0.99 | 1.00 | 1.00 | 0.99 | |
| test | 0.68 | 0.55-0.81 | 0.59 | 0.71 | 0.63 | 0.67 | ||
| LightGBM | train | 0.95 | 0.92-0.97 | 0.79 | 0.93 | 0.91 | 0.84 | |
| test | 0.74 | 0.61-0.86 | 0.69 | 0.68 | 0.65 | 0.72 | ||
| MLP | train | 0.84 | 0.79-0.88 | 0.64 | 0.84 | 0.77 | 0.74 | |
| test | 0.79 | 0.67-0.90 | 0.62 | 0.77 | 0.69 | 0.70 | ||
| DenseNet121 | SVM | train | 0.96 | 0.94-0.98 | 0.70 | 0.80 | 0.75 | 0.76 |
| test | 0.75 | 0.63-0.87 | 0.62 | 0.77 | 0.69 | 0.70 | ||
| KNN | train | 0.82 | 0.77-0.87 | 0.97 | 0.99 | 0.99 | 0.97 | |
| test | 0.72 | 0.60-0.85 | 0.38 | 0.79 | 0.61 | 0.60 | ||
| RandomForest | train | 1.00 | 1.00-1.00 | 1.00 | 1.00 | 1.00 | 1.00 | |
| test | 0.67 | 0.54-0.80 | 0.48 | 0.79 | 0.67 | 0.64 | ||
| ExtraTrees | train | 1.00 | 1.00-1.00 | 1.00 | 1.00 | 1.00 | 1.00 | |
| test | 0.69 | 0.56-0.82 | 0.69 | 0.68 | 0.65 | 0.72 | ||
| XGBoost | train | 1.00 | 1.00-1.00 | 0.79 | 0.93 | 0.90 | 0.84 | |
| test | 0.73 | 0.60-0.86 | 0.52 | 0.71 | 0.60 | 0.63 | ||
| LightGBM | train | 0.95 | 0.92-0.97 | 0.70 | 0.85 | 0.79 | 0.77 | |
| test | 0.72 | 0.59-0.84 | 0.62 | 0.74 | 0.67 | 0.69 | ||
| MLP | train | 0.88 | 0.83-0.92 | 0.70 | 0.80 | 0.75 | 0.76 | |
| test | 0.74 | 0.62-0.87 | 0.62 | 0.77 | 0.69 | 0.70 | ||
| AlexNet | SVM | train | 0.94 | 0.92-0.97 | 0.79 | 0.93 | 0.91 | 0.84 |
| test | 0.84 | 0.74-0.94 | 0.69 | 0.82 | 0.77 | 0.76 | ||
| KNN | train | 0.83 | 0.78-0.88 | 0.76 | 0.79 | 0.75 | 0.79 | |
| test | 0.62 | 0.47-0.76 | 0.62 | 0.59 | 0.56 | 0.65 | ||
| RandomForest | train | 1.00 | 1.00-1.00 | 0.97 | 1.00 | 1.00 | 0.97 | |
| test | 0.70 | 0.57-0.83 | 0.52 | 0.74 | 0.63 | 0.64 | ||
| ExtraTrees | train | 1.00 | 1.00-1.00 | 1.00 | 1.00 | 1.00 | 1.00 | |
| test | 0.59 | 0.45-0.73 | 0.31 | 0.71 | 0.47 | 0.55 | ||
| XGBoost | train | 1.00 | 1.00-1.00 | 0.99 | 1.00 | 1.00 | 0.99 | |
| test | 0.74 | 0.61-0.87 | 0.72 | 0.74 | 0.70 | 0.76 | ||
| LightGBM | train | 0.95 | 0.93-0.98 | 0.78 | 0.95 | 0.93 | 0.84 | |
| test | 0.62 | 0.48-0.77 | 0.59 | 0.71 | 0.63 | 0.67 | ||
| MLP | train | 0.87 | 0.82-0.91 | 0.73 | 0.88 | 0.84 | 0.80 | |
| test | 0.84 | 0.73-0.94 | 0.72 | 0.88 | 0.84 | 0.79 |