Table 3.

Quantification of learned shortcuts on the test set for the original classifier (“Vanilla”) and the retrained classifier trained on the training set where coloured patches are inpainted (“Retrained”). $\mathit{MAD}$: mean absolute deviation of predictions after removing resp. inserting patches. %$P\downarrow$: fraction of images where probability of true class decreases. $\mathit{MND}$: mean negative deviation. %flip: fraction of images where the prediction after removal/insertion crossed the decision boundary. Lower is better for all metrics.

Data Subsets	Classifier	$\mathit{MAD}$	%$\mathit{P}\downarrow$	$\mathit{MND}$	%flip
Benign with Patches Removed	Vanilla Classifier	0.268	99.7	0.268	21.1
	Retrained Classifier	0.044	40.4	0.047	0.6
Malignant with Patches Inserted	Vanilla Classifier	0.462	98.9	0.466	69.5
	Retrained Classifier	0.206	93.4	0.216	33.5