Figure 2.

Clinically relevant adversarial attacks on deep-learning systems for skin cancer diagnosis. Attacks were implemented against a pretrained Inception, version 3, network that was fine tuned for the differentiation of melanoma from benign melanocytic nevi. (a–j) Differential evolution-based adversarial attack through the modification of global color balance. (a) A schematic illustration of the differential evolution algorithm in addition to examples of (b, d, f, h) original and (c, e, g, i) perturbed images are shown. Green boxes indicate the confidence (i.e., the output of the network in favor of this class after softmax transformation) of the network in predicting melanoma for the original images, and red boxes indicate the confidence in the prediction of a benign nevus for the adversarial images. (j) Image illustrates the dependency of the successful adversarial attacks on initial classification by the network. For each image in the validation set, after the softmax transformation, the output of the final classification layer of the network is plotted for the original image (y-axis) versus the adversarial image (x-axis). (k–t) Differential evolution-based adversarial attack through the modification of image translation and rotation. (k) A schematic illustration of the differential evolution algorithm in addition to examples of (l, n, p, r) original and (m, o, q, s) perturbed images are shown along with the dependency of the successful adversarial attacks on the original classification by the (t) network plotted as in j. RGB, red green blue.